Course · 12 modules · 95 lessons · 672 min

Natural Language Processing

Text preprocessing, representation, sequence models, NLP tasks, information extraction, and multilingual NLP.

← All courses
Your progress0 / 95 lessons· 0%

The course at a glance

12 modules · click any tile to jump to its lessons.

All lessons

№ 01Foundations Of Language
01Ambiguity in LanguageLexical, syntactic, semantic, and referential ambiguity -- the core challenge that makes NLP hard, illustrated by why "I saw her duck" has at least five interpretations and how systems learn to resolve them.10 min→02Levels of Linguistic AnalysisThe hierarchy from phonology to pragmatics -- each level adding structure that NLP systems must capture to move from raw sound or text to genuine understanding.7 min→03MorphologyHow words are built from morphemes -- inflection, derivation, and compounding that affect meaning, and whose cross-linguistic variation profoundly shapes tokenization and NLP system design.7 min→04Pragmatics and DiscourseMeaning beyond the literal -- context, implicature, speech acts, Gricean maxims, and discourse coherence that determine what speakers actually communicate versus what their words technically say.8 min→05SemanticsThe study of meaning -- from word senses to compositional sentence meaning to logical representations, and the distributional hypothesis that reshaped how NLP systems learn what words mean.8 min→06Syntax and GrammarRules governing sentence structure -- phrase structure grammars, dependency relations, constituency trees, and the Chomsky hierarchy that defines the computational complexity of parsing.8 min→07Text as DataTreating language as structured data -- corpora, annotation schemes, inter-annotator agreement, and the paradigm shift from hand-crafted features to learned representations that transformed how NLP systems consume text.10 min→08What Is NLPNatural Language Processing is the interdisciplinary field at the intersection of linguistics, computer science, and artificial intelligence that enables machines to understand, generate, and reason about human language.7 min→
№ 02Text Preprocessing
01Data Annotation and LabelingCreating labeled NLP datasets through systematic annotation schemes, measuring inter-annotator agreement, managing crowdsourced labor, and applying active learning to minimize the high cost of human labeling.9 min→02Regular Expressions for NLPPattern matching as the workhorse of text preprocessing -- defining formal string patterns with a concise syntax to search, extract, validate, and transform text in NLP pipelines.6 min→03Sentence SegmentationDetecting sentence boundaries in running text despite the ambiguity of periods, which serve triple duty as sentence terminators, abbreviation markers, and decimal points.6 min→04Stemming and LemmatizationReducing words to base forms -- stemming by crude affix removal and lemmatization by linguistically-informed morphological analysis -- to collapse inflectional variants into shared representations.6 min→05Stopword RemovalFiltering high-frequency function words (the, is, at, which) that carry little semantic content to reduce noise and dimensionality in frequency-based text representations, though modern neural models often benefit from retaining them.6 min→06Text Cleaning and Noise RemovalHandling the messy reality of real-world text -- stripping HTML, fixing encoding errors, correcting OCR artifacts, normalizing social media conventions, deduplicating, and detecting language -- before any NLP model can be reliably applied.8 min→07Text NormalizationStandardizing text through case folding, unicode normalization, accent removal, and format unification so that superficially different strings map to a single canonical form before downstream processing.5 min→08Tokenization in NLPSplitting raw text into discrete units -- words and sentences -- using rule-based, statistical, or hybrid methods, with strategies that vary dramatically across languages and domains.6 min→
№ 03Text Representation
01Bag of WordsRepresenting text as unordered word frequency vectors -- simple, interpretable, and surprisingly effective for many classification and retrieval tasks.5 min→02Contextual EmbeddingsWord representations that change based on surrounding context -- the same word gets different vectors in different sentences, resolving polysemy and capturing nuance.7 min→03Document EmbeddingsRepresenting documents as dense vectors for retrieval, clustering, and classification at scale -- from TF-IDF with dimensionality reduction to neural encoders for long text.6 min→04FastTextSubword-aware embeddings that represent each word as the sum of its character n-gram vectors, gracefully handling morphology and out-of-vocabulary words.6 min→05GloVeGlobal matrix factorization of word co-occurrence statistics producing word vectors with linear substructures -- bridging count-based and prediction-based embedding methods.6 min→06N-Gram Language ModelsPredicting the next word from the previous N-1 words using maximum likelihood estimation -- the statistical foundation of language modeling.5 min→07Sentence EmbeddingsFixed-length vector representations of entire sentences -- from simple word vector averaging to dedicated neural encoders trained for semantic similarity.6 min→08TF-IDFWeighting words by term frequency times inverse document frequency to surface discriminative terms and suppress ubiquitous ones.5 min→09Word2VecLearning dense word vectors from co-occurrence via Skip-gram and CBOW -- the embedding revolution that showed words with similar meanings occupy nearby points in vector space.6 min→
№ 04Sequence Models
01Attention MechanismAttention allows a decoder to dynamically focus on different parts of the encoder's output at each generation step, replacing the fixed-size bottleneck vector with a weighted combination of all source representations.8 min→02Bidirectional RNNsBidirectional RNNs process a sequence in both forward and backward directions, producing representations that capture both past and future context at every time step.7 min→03Convolutional Models for TextCNNs applied to NLP use 1D convolutions over word embeddings to detect local n-gram patterns, offering parallelizable computation and strong performance for text classification, though with a limited receptive field compared to recurrent models.8 min→04Gated Recurrent UnitsGRUs simplify the LSTM gating mechanism by merging the cell state and hidden state into a single vector controlled by two gates, achieving comparable performance with fewer parameters.7 min→05Long Short-Term MemoryLSTMs introduce a gated cell state that acts as a controlled information highway, solving the vanishing gradient problem that cripples vanilla RNNs on long sequences.7 min→06Recurrent Neural NetworksRNNs process sequences one element at a time, maintaining a hidden state that accumulates information from previous time steps -- the first neural architecture designed for sequential data like language.6 min→07Sequence-to-Sequence ModelsThe encoder-decoder architecture maps variable-length input sequences to variable-length output sequences by compressing the input into a fixed-size context vector, then generating the output one token at a time.8 min→
№ 05Core Nlp Tasks Analysis
01Constituency ParsingConstituency parsing decomposes sentences into hierarchical phrase-structure trees, revealing how words group into nested constituents (noun phrases, verb phrases, etc.) according to a grammar.6 min→02Coreference ResolutionCoreference resolution determines which expressions (mentions) in a text refer to the same real-world entity, linking pronouns, names, and descriptions into coherent entity chains.5 min→03Dependency ParsingDependency parsing recovers the head-modifier relationships between words in a sentence, producing a tree structure that reveals grammatical dependencies without intermediate phrasal nodes.5 min→04Named Entity RecognitionNamed entity recognition (NER) identifies and classifies spans of text that refer to real-world entities such as persons, organizations, locations, dates, and other domain-specific categories.5 min→05Part-of-Speech TaggingPart-of-speech (POS) tagging assigns a grammatical category (noun, verb, adjective, etc.) to each word in a sentence, serving as a foundational sequence labeling task for downstream NLP.5 min→06Relation ExtractionRelation extraction identifies and classifies semantic relationships between entities mentioned in text, converting unstructured prose into structured knowledge triples (subject, relation, object).6 min→07Semantic Role LabelingSemantic role labeling (SRL) identifies the predicate-argument structure of a sentence -- determining who did what to whom, where, when, and how -- bridging syntactic parsing and meaning representation.5 min→08Sentiment AnalysisSentiment analysis detects subjective opinion, polarity (positive/negative/neutral), and emotion in text, operating at document, sentence, and aspect levels of granularity.5 min→09Text ClassificationText classification assigns one or more predefined category labels to a document, sentence, or passage, serving as the most widely deployed NLP capability in production systems.5 min→
№ 06Core Nlp Tasks Generation
01Data-to-Text GenerationConverting structured data (tables, knowledge graphs, database records) into fluent natural language descriptions, bridging the gap between databases and human-readable reports.6 min→02Dialogue SystemsConversational AI systems that interact with users through natural language, ranging from task-oriented slot-filling assistants to open-domain chatbots and modern LLM-based dialogue agents.7 min→03Grammatical Error CorrectionDetecting and correcting grammatical, spelling, and usage errors in written text, progressing from rule-based checkers through classifier ensembles to neural sequence-to-sequence and LLM-based approaches.7 min→04Machine TranslationAutomatically converting text from one human language to another, progressing from hand-crafted rules through statistical phrase tables to end-to-end neural models.6 min→05Paraphrase GenerationProducing semantically equivalent but syntactically different text, enabling data augmentation, style transfer, and deeper understanding of meaning.6 min→06Question AnsweringSystems that find or generate answers to natural language questions from given context, retrieved documents, or parametric knowledge.6 min→07Text GenerationProducing fluent, coherent text from a language model using decoding strategies that balance quality, diversity, and controllability.6 min→08Text SummarizationCondensing documents while preserving key information, using either extractive methods that select important sentences or abstractive methods that generate new condensed text.5 min→
№ 07Information Extraction And Retrieval
01Document SimilarityMeasuring how alike two documents are -- from lexical overlap measures like Jaccard and cosine similarity to semantic approaches like Word Mover's Distance and embedding-based comparison.8 min→02Event ExtractionIdentifying events, their triggers, and participant arguments from text -- detecting not just that something happened, but who was involved, where, when, and how.7 min→03Information ExtractionAutomatically extracting structured knowledge -- entities, relations, and events -- from unstructured text at scale, turning the flood of natural language into queryable data.6 min→04Information RetrievalFinding relevant documents from large collections in response to a user's information need -- from classical term-matching models like BM25 to modern neural dense retrieval.8 min→05Keyword ExtractionIdentifying the most important terms and phrases that characterize a document's content -- from statistical frequency methods to graph-based and embedding-based approaches.6 min→06Knowledge Graphs for NLPStructured knowledge representations connecting entities and relations in graph form -- enabling reasoning, retrieval, and grounding that complement the statistical patterns learned by language models.7 min→07Open Information ExtractionExtracting relation triples from text without predefined schemas -- domain-independent knowledge harvesting that scales across the open web.7 min→08Topic ModelingDiscovering latent themes in document collections by learning probabilistic or algebraic decompositions that map documents to topic mixtures and topics to word distributions.7 min→
№ 08Semantic Understanding
01Aspect-Based Sentiment AnalysisAspect-based sentiment analysis (ABSA) goes beyond document-level opinion mining to identify specific aspects of entities and the sentiment expressed toward each, enabling fine-grained understanding of opinions like "The food was great but the service was terrible."10 min→02Commonsense ReasoningCommonsense reasoning is the ability to draw on implicit world knowledge that humans take for granted -- physical intuitions, social conventions, and causal expectations -- to understand and reason about language.9 min→03Natural Language InferenceNatural language inference (NLI) classifies the relationship between a premise and hypothesis as entailment, contradiction, or neutral, serving as both a core semantic reasoning benchmark and a versatile tool for zero-shot NLP.9 min→04Negation and Speculation DetectionNegation and speculation detection identifies negated and uncertain statements in text -- determining not just what is said, but what is denied or merely hypothesized -- a capability critical for biomedical NLP, information extraction, and sentiment analysis.10 min→05Semantic SimilaritySemantic similarity measures the degree of meaning overlap between two linguistic units -- words, sentences, or documents -- providing a graded, continuous score rather than a categorical judgment.8 min→06Temporal ReasoningTemporal reasoning is the ability to identify, interpret, and reason about time expressions, event ordering, and temporal relationships in text, enabling systems to construct timelines and answer when-questions.10 min→07Textual EntailmentTextual entailment is the task of determining whether the meaning of one text (the hypothesis) can be logically inferred from another text (the premise), forming the foundation of computational semantic inference.8 min→08Word Sense DisambiguationWord sense disambiguation (WSD) is the task of determining which meaning of a polysemous word is intended in a given context, resolving one of the oldest and most fundamental ambiguities in natural language processing.8 min→
№ 09Pre Trained Models For Nlp
01BERTBERT (Bidirectional Encoder Representations from Transformers) pre-trains a deep transformer encoder using masked language modeling and next sentence prediction, producing bidirectional contextualized representations that shattered records across 11 NLP benchmarks and spawned an entire family of variants that continue to dominate NLP.7 min→02Cross-Lingual TransferCross-lingual transfer leverages multilingual pre-trained models to transfer NLP capabilities from high-resource languages (primarily English) to low-resource languages without target-language labeled data -- enabling zero-shot task performance across 100+ languages through shared representations.7 min→03Domain AdaptationDomain adaptation extends general-purpose pre-trained models to specialized domains -- biomedical, scientific, financial, legal, and clinical text -- through continued pre-training on domain corpora, producing models like BioBERT, SciBERT, and FinBERT that outperform their general counterparts by 2-10% on in-domain tasks.8 min→04ELMoELMo (Embeddings from Language Models) produces deep contextualized word representations by running a two-layer bidirectional LSTM language model, generating different vectors for the same word depending on its surrounding context -- the first major pre-trained model that bridged static word embeddings and modern transformers.7 min→05GPT for NLP TasksThe GPT series -- from GPT-1's generative pre-training with discriminative fine-tuning, through GPT-2's surprising zero-shot abilities, to GPT-3's in-context learning revolution -- demonstrated that autoregressive decoder-only transformers can perform virtually any NLP task through prompting alone, without task-specific fine-tuning.7 min→06Prompt-Based NLPPrompt-based NLP reformulates traditional NLP tasks as cloze-style fill-in-the-blank or text generation problems, leveraging pre-trained language models' existing knowledge to perform tasks with minimal or zero labeled data by converting classification into "predict the next/masked word" problems.8 min→07T5 and Text-to-TextT5 (Text-to-Text Transfer Transformer) unifies every NLP task -- classification, translation, summarization, question answering, and more -- into a single text-to-text framework where both inputs and outputs are text strings, enabling a systematic comparison of pre-training objectives, architectures, and datasets at scales from 60M to 11B parameters.8 min→08Transfer Learning in NLPTransfer learning revolutionized NLP by replacing task-specific training from scratch with a two-stage paradigm -- pre-train on massive unlabeled corpora, then fine-tune on small task-specific datasets -- reducing data requirements by 10-100x and establishing new state-of-the-art results across virtually every benchmark.7 min→
№ 10Multilingual And Low Resource Nlp
01Cross-Lingual Word EmbeddingsAligning word vector spaces from different languages into a shared space so that "cat" in English and "gato" in Spanish occupy nearby points -- enabling cross-lingual transfer without parallel corpora.6 min→02Data Augmentation for NLPGenerating synthetic training examples through techniques like back-translation, synonym replacement, and contextual generation to improve model performance when labeled data is scarce -- typically yielding 5--30% improvements depending on baseline data size.8 min→03Language Diversity and TypologyHow the structural properties of the world's languages -- word order, morphological complexity, and writing systems -- create distinct challenges for NLP systems that are overwhelmingly designed for English.10 min→04Low-Resource NLPTechniques for building effective NLP systems when labeled data is scarce -- from few-shot and zero-shot learning to active learning and cross-lingual transfer -- addressing the reality that most languages and domains lack sufficient annotated data.8 min→05Machine Translation ApproachesThe evolution of machine translation from hand-coded linguistic rules through statistical phrase tables to end-to-end neural models -- each paradigm shift dramatically improving quality and reducing engineering effort.8 min→06Multilingual NLPBuilding NLP systems that work across multiple languages -- navigating the tension between universal representations and the enormous diversity of the world's 7,000+ languages.7 min→07Multilingual TransformersPre-trained transformer models like mBERT and XLM-R that learn shared representations across 100+ languages from massive multilingual corpora, enabling zero-shot cross-lingual transfer.7 min→
№ 11Speech And Multimodal Nlp
01Automatic Speech RecognitionConverting spoken language into written text by mapping acoustic signals through feature extraction, acoustic modeling, and language decoding -- progressing from HMM-GMM pipelines to end-to-end neural systems like Whisper.7 min→02Document UnderstandingExtracting and understanding information from visually rich documents (forms, invoices, reports, tables) by jointly modeling text content, visual appearance, and spatial layout -- powered by the LayoutLM family and multimodal document representations.8 min→03Image CaptioningGenerating natural language descriptions of images by bridging visual perception and language generation -- from CNN-LSTM pipelines to attention-based and transformer models, now increasingly subsumed by vision-language foundation models.8 min→04Multimodal NLPCombining language with vision, audio, and other modalities to build systems that perceive and reason across multiple information channels -- from contrastive pre-training (CLIP) to multimodal large language models (GPT-4V, Gemini).7 min→05Speech Language ModelsUnified models that process both text and speech as token sequences, enabling zero-shot voice cloning, speech generation, and the convergence toward universal language models that handle any modality.8 min→06Text-to-SpeechGenerating natural-sounding human speech from written text, progressing from concatenative and parametric methods to neural systems (Tacotron, WaveNet, FastSpeech) that approach human-level naturalness.7 min→07Visual Question AnsweringAnswering natural language questions about images by jointly reasoning over visual and textual information -- a fundamental test of multimodal understanding that exposes the tension between genuine reasoning and superficial language bias.8 min→
№ 12Evaluation And Ethics
01Bias in NLPNLP systems absorb, reproduce, and often amplify societal biases present in training data, annotation practices, and modeling decisions, leading to systematic disadvantages for underrepresented groups.8 min→02Evaluation Metrics for NLPAutomated evaluation metrics quantify NLP system performance using formulas that approximate human judgment, each capturing a different facet of quality -- from exact-match precision to semantic embedding similarity.6 min→03Fairness in NLPFairness in NLP formalizes the requirement that language technologies perform equitably across demographic groups, using mathematical definitions that reveal fundamental trade-offs between competing notions of what "fair" means.8 min→04Human Evaluation for NLPHuman evaluation remains the gold standard for assessing NLP system quality, using structured protocols with trained annotators to judge dimensions -- fluency, adequacy, coherence -- that automated metrics cannot reliably capture.7 min→05Intrinsic vs. Extrinsic EvaluationIntrinsic evaluation measures a model component's quality in isolation (e.g., perplexity for a language model), while extrinsic evaluation measures its contribution to a downstream end-task (e.g., translation accuracy).7 min→06NLP for Social GoodNLP technologies can address critical societal challenges -- from extracting life-saving information from clinical notes to preserving endangered languages -- when designed with care for the communities they serve.8 min→07Privacy in NLPLanguage models memorize and can regurgitate sensitive training data -- including personal identifiers, phone numbers, and medical records -- creating privacy risks that require techniques like differential privacy, federated learning, and de-identification to mitigate.9 min→08Responsible NLP DevelopmentResponsible NLP development encompasses the practices, documentation standards, and ethical frameworks -- from model cards to carbon footprint accounting -- that ensure language technologies are built, evaluated, and deployed with transparency, accountability, and awareness of potential harms.9 min→