Home > Research > Publications & Outputs > A Data-driven Latent Semantic Analysis for Auto...

Associated organisational unit

Electronic data

  • Accepted_Version_BigD220

    Rights statement: ©2023 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. Requirements for research articles as indicated by UKRI Open Access Policy requirements for in-scope research articles: We follow ROUTE 1: Publish the research article open access in a journal or publishing platform which makes the Version of Record immediately open access via its website; The Version of Record must be free and unrestricted to view and download. It must have a Creative Commons Attribution (CC BY) licence, or other licence permitted by UKRI.

    Accepted author manuscript, 8.75 MB, PDF document

    Available under license: CC BY: Creative Commons Attribution 4.0 International License

Links

Text available via DOI:

View graph of relations

A Data-driven Latent Semantic Analysis for Automatic Text Summarization using LDA Topic Modelling

Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSNConference contribution/Paperpeer-review

Published
Close
Publication date26/01/2023
Host publication2022 IEEE International Conference on Big Data (Big Data)
PublisherIEEE
Pages2771-2780
Number of pages10
ISBN (electronic)9781665480451
ISBN (print)9781665480468
<mark>Original language</mark>English

Publication series

Name2022 IEEE International Conference on Big Data (Big Data)
PublisherIEEE

Abstract

With the advent and popularity of big data mining and huge text analysis in modern times, automated text summarization became prominent for extracting and retrieving important information from documents. This research investigates aspects of automatic text summarization from the perspectives of single and multiple documents. Summarization is a task of condensing huge text articles into short, summarized versions. The text is reduced in size for summarization purpose but preserving key vital information and retaining the meaning of the original document. This study presents the Latent Dirichlet Allocation (LDA) approach used to perform topic modelling from summarised medical science journal articles with topics related to genes and diseases. In this study, PyLDAvis web-based interactive visualization tool was used to visualise the selected topics. The visualisation provides an overarching view of the main topics while allowing and attributing deep meaning to the prevalence individual topic. This study presents a novel approach to summarization of single and multiple documents. The results suggest the terms ranked purely by considering their probability of the topic prevalence within the processed document using extractive summarization technique. PyLDAvis visualization describes the flexibility of exploring the terms of the topics’ association to the fitted LDA model. The topic modelling result shows prevalence within topics 1 and 2. This association reveals that there is similarity between the terms in topic 1 and 2 in this study. The efficacy of the LDA and the extractive summarization methods were measured using Latent Semantic Analysis (LSA) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics to evaluate the reliability and validity of the model.

Bibliographic note

©2023 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. Requirements for research articles as indicated by UKRI Open Access Policy requirements for in-scope research articles: We follow ROUTE 1: Publish the research article open access in a journal or publishing platform which makes the Version of Record immediately open access via its website; The Version of Record must be free and unrestricted to view and download. It must have a Creative Commons Attribution (CC BY) licence, or other licence permitted by UKRI.