USF Librarian Contacts

Text & Data Mining

Text Mining, Data Mining, and Computational Text Analysis

What is TDM?

Text data mining or text and data mining (TDM) is an automated method of processing and analyzing vast quantities of digital information. Text, data, and web mining are related, perhaps nested, methods for finding patterns and extracting valuable knowledge from data sets.

Cai, Y., Sun, JT. (2009). Text Mining. In: LIU, L., ÖZSU, M.T. (eds) Encyclopedia of Database Systems. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-39940-9_418
Anandarajan M., Hill C., Nolan T. (2019) Introduction to Text Analytics. In: Practical Text Analytics. Advances in Analytics and Data Science, vol 2. Springer, Cham. https://doi.org/10.1007/978-3-319-95663-3_1

TDM and AI

The TDM process can be broken down into three steps: 1) Access, 2) Extraction, 3) Mining. Non-generative AI has been used in text and data mining projects for many years. Many text and data mining processes now incorporate generative AI to enhance the process of deriving new meaning from mined text.

Some database vendors and publishers to which the Libraries subscribe differentiate between TDM that is either non-generative or that has the end goal of content analysis and TDM that is used as an initial process of a project to train a generative AI tool or agent to perform an additional task. Pay close attention to the terms and conditions of API services and databases. Additional permissions may be necessary for certain projects.