HomeDeep LearningDeep Learning for Text Mining from Scratch

Here is a list of courses or materials for you to learn deep learning for text mining from scratch。


Everything start from mathematics.

1. Pre-Calculus

About the Course
Through this course, students will acquire a solid foundation in algebra and trigonometry. The course concentrates on the various functions that are important to the study of the calculus. Emphasis is placed on understanding the properties of linear, polynomial, piecewise, exponential, logarithmic and trigonometric functions. Students will learn to work with various types of functions in symbolic, graphical, numerical and verbal form.

2. Calculus: Single Variable

About the Course
Calculus is one of the grandest achievements of human thought, explaining everything from planetary orbits to the optimal size of a city to the periodicity of a heartbeat. This brisk course covers the core ideas of single-variable Calculus with emphases on conceptual understanding and applications. The course is ideal for students beginning in the engineering, physical, and social sciences. Distinguishing features of the course include:

the introduction and use of Taylor series and approximations from the beginning;
a novel synthesis of discrete and continuous forms of Calculus;
an emphasis on the conceptual over the computational; and
a clear, dynamic, unified approach.

3. Calculus One

About this Course
Calculus is about the very large, the very small, and how things change. The surprise is that something seemingly so abstract ends up explaining the real world. Calculus plays a starring role in the biological, physical, and social sciences. By focusing outside of the classroom, we will see examples of calculus appearing in daily life.

This course is a first and friendly introduction to calculus, suitable for someone who has never seen the subject before, or for someone who has seen some calculus but wants to review the concepts and practice applying those concepts to solve problems.

4. Calculus Two: Sequences and Series

About this Course
Calculus Two: Sequences and Series is an introduction to sequences, infinite series, convergence tests, and Taylor series. The course emphasizes not just getting answers, but asking the question “why is this true?”

5. Introduction to Probability – The Science of Uncertainty

About this course

The world is full of uncertainty: accidents, storms, unruly financial markets, noisy communications. The world is also full of data. Probabilistic modeling and the related field of statistical inference are the keys to analyzing data and making scientifically sound predictions.

Probabilistic models use the language of mathematics. But instead of relying on the traditional “theorem – proof” format, we develop the material in an intuitive — but still rigorous and mathematically precise — manner. Furthermore, while the applications are multiple and evident, we emphasize the basic concepts and methodologies that are universally applicable.

The course covers all of the basic probability concepts, including:

multiple discrete or continuous random variables, expectations, and conditional distributions
laws of large numbers
the main tools of Bayesian inference methods
an introduction to random processes (Poisson processes and Markov chains)

6. Statistics: The Science of Decisions

We live in a time of unprecedented access to information…data. Whether researching the best school, job, or relationship, the Internet has thrown open the doors to vast pools of data. Statistics are simply objective and systematic methods for describing and interpreting information so that you may make the most informed decisions about life.

7. Linear Algebra – Foundations to Frontiers

About this course
Foundations to Frontiers (LAFF) is packed full of challenging, rewarding material that is essential for mathematicians, engineers, scientists, and anyone working with large datasets. Students appreciate our unique approach to teaching linear algebra because:

It’s visual.
It connects hand calculations, mathematical abstractions, and computer programming.
It illustrates the development of mathematical theory.
It’s applicable.
In this course, you will learn all the standard topics that are taught in typical undergraduate linear algebra courses all over the world, but using our unique method, you’ll also get more! LAFF was developed following the syllabus of an introductory linear algebra course at The University of Texas at Austin taught by Professor Robert van de Geijn, an expert on high performance linear algebra libraries. Through short videos, exercises, visualizations, and programming assignments, you will study Vector and Matrix Operations, Linear Transformations, Solving Systems of Equations, Vector Spaces, Linear Least-Squares, and Eigenvalues and Eigenvectors. In addition, you will get a glimpse of cutting edge research on the development of linear algebra libraries, which are used throughout computational science.

8. Linear Algebra by Prof. Gilbert Strang

Course Description
This is a basic subject on matrix theory and linear algebra. Emphasis is given to topics that will be useful in other disciplines, including systems of equations, vector spaces, determinants, eigenvalues, similarity, and positive definite matrices.

9. Convex Optimization


This course concentrates on recognizing and solving convex optimization problems that arise in applications. The syllabus includes: convex sets, functions, and optimization problems; basics of convex analysis; least-squares, linear and quadratic programs, semidefinite programming, minimax, extremal volume, and other problems; optimality conditions, duality theory, theorems of alternative, and applications; interior-point methods; applications to signal processing, statistics and machine learning, control and mechanical engineering, digital and analog circuit design, and finance.


You should have good knowledge of linear algebra and exposure to probability. Exposure to numerical computing, optimization, and application fields is helpful but not required; the applications will be kept basic and simple. You will use matlab and CVX to write simple scripts, so some basic familiarity with matlab is helpful. We will provide some basic Matlab tutorials.

Programming Using Python

Python is one of the best programming choice for beginner.

10. Introduction to Computer Science

Course Summary
In this introduction to computer programming course, you’ll learn and practice key computer science concepts by building your own versions of popular web applications. You’ll learn Python, a powerful, easy-to-learn, and widely used programming language, and you’ll explore computer science basics, as you build your own search engine and social network.

11. Learn to Program: The Fundamentals

About the Course
A computer program is a set of instructions for a computer to follow, just as a recipe is a set of instructions for a chef. Laptops, kitchen appliances, MP3 players, and many other electronic devices all run computer programs. Programs have been written to manipulate sound and video, write poetry, run banking systems, predict the weather, and analyze athletic performance. This course is intended for people who have never seen a computer program. It will give you a better understanding of how computer applications work and teach you how to write your own applications. More importantly, you’ll start to learn computational thinking, which is a fundamental approach to solving real-world problems. Computer programming languages share common fundamental concepts, and this course will introduce you to those concepts using the Python programming language. By the end of this course, you will be able to write your own programs to process data from the web and create interactive text-based games.

12. Introduction to Computer Science and Programming Using Python

About this course
This course is the first of a two-course sequence: Introduction to Computer Science and Programming Using Python, and Introduction to Computational Thinking and Data Science. Together, they are designed to help people with no prior exposure to computer science or programming learn to think computationally and write programs to tackle useful problems. Some of the people taking the two courses will use them as a stepping stone to more advanced computer science courses, but for many it will be their first and last computer science courses.

Since these courses may be the only formal computer science courses many of the students take, we have chosen to focus on breadth rather than depth. The goal is to provide students with a brief introduction to many topics so they will have an idea of what is possible when they need to think about how to use computation to accomplish some goal later in their career. That said, they are not “computation appreciation” courses. They are challenging and rigorous courses in which the students spend a lot of time and effort learning to bend the computer to their will.

13. Algorithms: Design and Analysis, Part 1

About the Course
In this course you will learn several fundamental principles of algorithm design. You’ll learn the divide-and-conquer design paradigm, with applications to fast sorting, searching, and multiplication. You’ll learn several blazingly fast primitives for computing on graphs, such as how to compute connectivity information and shortest paths. Finally, we’ll study how allowing the computer to “flip coins” can lead to elegant and practical algorithms and data structures. Learn the answers to questions such as: How do data structures like heaps, hash tables, bloom filters, and balanced search trees actually work, anyway? How come QuickSort runs so fast? What can graph algorithms tell us about the structure of the Web and social networks? Did my 3rd-grade teacher explain only a suboptimal algorithm for multiplying two numbers?

14. Algorithms: Design and Analysis, Part 2

About the Course
In this course you will learn several fundamental principles of advanced algorithm design. You’ll learn the greedy algorithm design paradigm, with applications to computing good network backbones (i.e., spanning trees) and good codes for data compression. You’ll learn the tricky yet widely applicable dynamic programming algorithm design paradigm, with applications to routing in the Internet and sequencing genome fragments. You’ll learn what NP-completeness and the famous “P vs. NP” problem mean for the algorithm designer. Finally, we’ll study several strategies for dealing with hard (i.e., NP-complete problems), including the design and analysis of heuristics. Learn how shortest-path algorithms from the 1950s (i.e., pre-ARPANET!) govern the way that your Internet traffic gets routed today; why efficient algorithms are fundamental to modern genomics; and how to make a million bucks in prize money by “just” solving a math problem!

Machine leaning

Learning deep learning from basic machine learning course is very necessary, here is some cool machine learning courses you should learn.

15. Machine Learning by Andrew Ng from Stanford University on Coursera

Maybe this is the most popular machine learning course on the internet, now is one of the self-faced course on the Coursera and you can learn it any time.

About this Course

Machine learning is the science of getting computers to act without being explicitly programmed. In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it. Many researchers also think it is the best way to make progress towards human-level AI. In this class, you will learn about the most effective machine learning techniques, and gain practice implementing them and getting them to work for yourself. More importantly, you’ll learn about not only the theoretical underpinnings of learning, but also gain the practical know-how needed to quickly and powerfully apply these techniques to new problems. Finally, you’ll learn about some of Silicon Valley’s best practices in innovation as it pertains to machine learning and AI.

This course provides a broad introduction to machine learning, datamining, and statistical pattern recognition. Topics include: (i) Supervised learning (parametric/non-parametric algorithms, support vector machines, kernels, neural networks). (ii) Unsupervised learning (clustering, dimensionality reduction, recommender systems, deep learning). (iii) Best practices in machine learning (bias/variance theory; innovation process in machine learning and AI). The course will also draw from numerous case studies and applications, so that you’ll also learn how to apply learning algorithms to building smart robots (perception, control), text understanding (web search, anti-spam), computer vision, medical informatics, audio, database mining, and other areas.

16. Machine Learning by Pedro Domingos from University of Washington on Coursera

Actually this course is never opened on Coursera, but you can watch it by preview lectures.

About the Course
Machine learning algorithms can figure out how to perform important tasks by generalizing from examples. This is often feasible and cost-effective when manual programming is not. Machine learning (also known as data mining, pattern recognition and predictive analytics) is used widely in business, industry, science and government, and there is a great shortage of experts in it. If you pick up a machine learning textbook you may find it forbiddingly mathematical, but in this class you will learn that the key ideas and algorithms are in fact quite intuitive. And powerful!

Most of the class will be devoted to supervised learning (in other words, learning in which a teacher provides the learner with the correct answers at training time). This is the most mature and widely used type of machine learning. We will cover the main supervised learning techniques, including decision trees, rules, instances, Bayesian techniques, neural networks, model ensembles, and support vector machines. We will also touch on learning theory with an emphasis on its practical uses. Finally, we will cover the two main classes of unsupervised learning methods: clustering and dimensionality reduction. Throughout the class there will be an emphasis not just on individual algorithms but on ideas that cut across them and tips for making them work.

In the class projects you will build your own implementations of machine learning algorithms and apply them to problems like spam filtering, clickstream mining, recommender systems, and computational biology. This will get you as close to becoming a machine learning expert as you can in ten weeks!

17. Learning From Data

“A real Caltech course, not a watered-down version”

This is an introductory course in machine learning (ML) that covers the basic theory, algorithms, and applications. ML is a key technology in Big Data, and in many financial, medical, commercial, and scientific applications. It enables computational systems to adaptively improve their performance with experience accumulated from the observed data. ML has become one of the hottest fields of study today, taken up by undergraduate and graduate students from 15 different majors at Caltech. This course balances theory and practice, and covers the mathematical as well as the heuristic aspects. The lectures below follow each other in a story-like fashion:
What is learning?
Can a machine learn?
How to do it?
How to do it well?
Take-home lessons.

Deep Learning

18. Neural Networks for Machine Learning

This course is taught by Geoffrey Hinton since 2012, and definitely first choice of deep learning and neural networks guide:

About the Course
Neural networks use learning algorithms that are inspired by our understanding of how the brain learns, but they are evaluated by how well they work for practical applications such as speech recognition, object recognition, image retrieval and the ability to recommend products that a user will like. As computers become more powerful, Neural Networks are gradually taking over from simpler Machine Learning methods. They are already at the heart of a new generation of speech recognition devices and they are beginning to outperform earlier systems for recognizing objects in images. The course will explain the new learning procedures that are responsible for these advances, including effective new proceduresr for learning multiple layers of non-linear features, and give you the skills and understanding required to apply these procedures in many other domains.

19. UFLDL Tutorial

Description: This tutorial will teach you the main ideas of Unsupervised Feature Learning and Deep Learning. By working through it, you will also get to implement several feature learning/deep learning algorithms, get to see them work for yourself, and learn how to apply/adapt these ideas to new problems.
This tutorial assumes a basic knowledge of machine learning (specifically, familiarity with the ideas of supervised learning, logistic regression, gradient descent). If you are not familiar with these ideas, we suggest you go to this Machine Learning course and complete sections II, III, IV (up to Logistic Regression) first.

Text Mining or Natural Language Processing

20. Natural Language Processing by Dan Jurafsky and Christopher Manning from Stanford University on Coursera

About the Course
This course covers a broad range of topics in natural language processing, including word and sentence tokenization, text classification and sentiment analysis, spelling correction, information extraction, parsing, meaning extraction, and question answering, We will also introduce the underlying theory from probability, statistics, and machine learning that are crucial for the field, and cover fundamental algorithms like n-gram language modeling, naive bayes and maxent classifiers, sequence models like Hidden Markov Models, probabilistic dependency and constituent parsing, and vector-space models of meaning.

We are offering this course on Natural Language Processing free and online to students worldwide, continuing Stanford’s exciting forays into large scale online instruction. Students have access to screencast lecture videos, are given quiz questions, assignments and exams, receive regular feedback on progress, and can participate in a discussion forum. Those who successfully complete the course will receive a statement of accomplishment. Taught by Professors Jurafsky and Manning, the curriculum draws from Stanford’s courses in Natural Language Processing. You will need a decent internet connection for accessing course materials, but should be able to watch the videos on your smartphone.

21. Natural Language Processing by Michael Collins from Columbia University on Coursera

About the Course
Natural language processing (NLP) deals with the application of computational models to text or speech data. Application areas within NLP include automatic (machine) translation between languages; dialogue systems, which allow a human to interact with a machine using natural language; and information extraction, where the goal is to transform unstructured text into structured (database) representations that can be searched and browsed in flexible ways. NLP technologies are having a dramatic impact on the way people interact with computers, on the way people interact with each other through the use of language, and on the way people access the vast amount of linguistic data now in electronic form. From a scientific viewpoint, NLP involves fundamental questions of how to structure formal models (for example statistical models) of natural language phenomena, and of how to design algorithms that implement these models.

In this course you will study mathematical and computational models of language, and the application of these models to key problems in natural language processing. The course has a focus on machine learning methods, which are widely used in modern NLP systems: we will cover formalisms such as hidden Markov models, probabilistic context-free grammars, log-linear models, and statistical models for machine translation. The curriculum closely follows a course currently taught by Professor Collins at Columbia University, and previously taught at MIT.

22. Text Mining and Analytics by ChengXiang Zhai from University of Illinois at Urbana-Champaign on Coursera

About the Course
This course will cover the major techniques for mining and analyzing text data to discover interesting patterns, extract useful knowledge, and support decision making, with an emphasis on statistical approaches that can be generally applied to arbitrary text data in any natural language with no or minimum human effort.

Detailed analysis of text data requires understanding of natural language text, which is known to be a difficult task for computers. However, a number of statistical approaches have been shown to work well for the “shallow” but robust analysis of text data for pattern finding and knowledge discovery. You will learn the basic concepts, principles, and major algorithms in text mining and their potential applications.

23. Introduction to Natural Language Processing by Dragomir R. Radev from University of Michigan on Coursera

About the Course
This course provides an introduction to the field of Natural Language Processing. It includes relevant background material in Linguistics, Mathematics, Statistics, and Computer Science. Some of the topics covered in the class are Text Similarity, Part of Speech Tagging, Parsing, Semantics, Question Answering, Sentiment Analysis, and Text Summarization.
The course includes quizzes, programming assignments in python, and a final exam.

Deep Learning for Text Mining or DeepNLP

24. ACL 2012 + NAACL 2013 Tutorial: Deep Learning for NLP (without Magic) by Richard Socher, Chris Manning and Yoshua Bengio


Machine learning is everywhere in today’s NLP, but by and large machine learning amounts to numerical optimization of weights for human designed representations and features. The goal of deep learning is to explore how computers can take advantage of data to develop features and representations appropriate for complex interpretation tasks. This tutorial aims to cover the basic motivation, ideas, models and learning algorithms in deep learning for natural language processing. Recently, these methods have been shown to perform very well on various NLP tasks such as language modeling, POS tagging, named entity recognition, sentiment analysis and paraphrase detection, among others. The most attractive quality of these techniques is that they can perform well without any external hand-designed resources or time-intensive feature engineering. Despite these advantages, many researchers in NLP are not familiar with these methods. Our focus is on insight and understanding, using graphical illustrations and simple, intuitive derivations. The goal of the tutorial is to make the inner workings of these techniques transparent, intuitive and their results interpretable, rather than black boxes labeled “magic here”. The first part of the tutorial presents the basics of neural networks, neural word vectors, several simple models based on local windows and the math and algorithms of training via backpropagation. In this section applications include language modeling and POS tagging. In the second section we present recursive neural networks which can learn structured tree outputs as well as vector representations for phrases and sentences. We cover both equations as well as applications. We show how training can be achieved by a modified version of the backpropagation algorithm introduced before. These modifications allow the algorithm to work on tree structures. Applications include sentiment analysis and paraphrase detection. We also draw connections to recent work in semantic compositionality in vector spaces. The principle goal, again, is to make these methods appear intuitive and interpretable rather than mathematically confusing. By this point in the tutorial, the audience members should have a clear understanding of how to build a deep learning system for word-, sentence- and document-level tasks. The last part of the tutorial gives a general overview of the different applications of deep learning in NLP, including bag of words models. We will provide a discussion of NLP-oriented issues in modeling, interpretation, representational power, and optimization.

25. Deep Learning for Natural Language Processing by Richard Socher from Stanford University

Course Description
Natural language processing (NLP) is one of the most important technologies of the information age. Understanding complex language utterances is also a crucial part of artificial intelligence. Applications of NLP are everywhere because people communicate most everything in language: web search, advertisement, emails, customer service, language translation, radiology reports, etc. There are a large variety of underlying tasks and machine learning models powering NLP applications. Recently, deep learning approaches have obtained very high performance across many different NLP tasks. These models can often be trained with a single end-to-end model and do not require traditional, task-specific feature engineering. In this spring quarter course students will learn to implement, train, debug, visualize and invent their own neural network models. The course provides a deep excursion into cutting-edge research in deep learning applied to NLP. The final project will involve training a complex recurrent neural network and applying it to a large scale NLP problem. On the model side we will cover word vector representations, window-based neural networks, recurrent neural networks, long-short-term-memory models, recursive neural networks, convolutional neural networks as well as some very novel models involving a memory component. Through lectures and programming assignments students will learn the necessary engineering tricks for making neural networks work on practical problems.

Posted by TextMiner


Deep Learning for Text Mining from Scratch — No Comments

Leave a Reply

Your email address will not be published. Required fields are marked *