Actions for Pretrained transformers for text ranking : BERT and beyond

Email RIS file
Bookmark
Report an Issue

Pretrained transformers for text ranking : BERT and beyond / Jimmy Lin, Rodrigo Nogueira, Andrew Yates

Author
Lin, Jimmy, 1979-
Published
San Rafael, California (1537 Fourth Street, 1537 Fourth Street, San Rafael, CA 94901 USA) : Morgan & Claypool Publishers, [2022]
Physical Description
1 PDF (xiii, 307 pages) : color illustrations
Additional Creators
Nogueira, Rodrigo and Yates, Andrew
Access Online

Availability

I Want It
I Want It

Finding items...

Series
Restrictions on Access
Abstract freely available; full-text restricted to subscribers or individual document purchasers.
Contents
1. Introduction -- 1.1. Text ranking problems -- 1.2. A brief history -- 1.3. Roadmap, assumptions, and omissions, 2. Setting the stage -- 2.1. Texts -- 2.2. Information needs -- 2.3. Relevance -- 2.4. Relevance judgments -- 2.5. Ranking metrics -- 2.6. Community evaluations and reusable test collections -- 2.7. Descriptions of common test collections -- 2.8. Keyword search -- 2.9. Notes on parlance, 3. Multi-stage architectures for reranking -- 3.1. A high-level overview of BERT -- 3.2. Simple relevance classification : monoBERT -- 3.3. From passage to document ranking -- 3.4. From single-stage to multi-stage rerankers -- 3.5. Beyond BERT -- 3.6. Concluding thoughts, 4. Refining query and document representations -- 4.1. Query and document expansion : general remarks -- 4.2. Pseudo-relevance feedback with contextualized embeddings : CEQE -- 4.3. Document expansion via query prediction : doc2query -- 4.4. Term reweighting as regression : DeepCT -- 4.5. Term reweighting with weak supervision : HDCT -- 4.6. Combining term expansion with term weighting : DeepImpact -- 4.7. Expansion of query and document representations -- 4.8. Concluding thoughts, 5. Learned dense representations for ranking -- 5.1. Task formulation -- 5.2. Nearest neighbor search -- 5.3. Pre-BERT text representations for ranking -- 5.4. Simple transformer bi-encoders for ranking -- 5.5. Enhanced transformer bi-encoders for ranking -- 5.6. Knowledge distillation for transformer bi-encoders -- 5.7. Concluding thoughts, and 6. Future directions and conclusions -- 6.1. Notable content omissions -- 6.2. Open research questions -- 6.3. Final thoughts.
Summary
The goal of text ranking is to generate an ordered list of texts retrieved from a corpus in response to a query. Although the most common formulation of text ranking is search, instances of the task can also be found in many natural language processing (NLP) applications.This book provides an overview of text ranking with neural network architectures known as transformers, of which BERT (Bidirectional Encoder Representations from Transformers) is the best-known example. The combination of transformers and self-supervised pretraining has been responsible for a paradigm shift in NLP, information retrieval (IR), and beyond. This book provides a synthesis of existing work as a single point of entry for practitioners who wish to gain a better understanding of how to apply transformers to text ranking problems and researchers who wish to pursue work in this area. It covers a wide range of modern techniques, grouped into two high-level categories: transformer models that perform reranking in multi-stage architectures and dense retrieval techniques that perform ranking directly. Two themes pervade the book: techniques for handling long documents, beyond typical sentence-by-sentence processing in NLP, and techniques for addressing the tradeoff between effectiveness (i.e., result quality) and efficiency (e.g., query latency, model and index size). Although transformer architectures and pretraining techniques are recent innovations, many aspects of how they are applied to text ranking are relatively well understood and represent mature techniques. However, there remain many open research questions, and thus in addition to laying out the foundations of pretrained transformers for text ranking, this book also attempts to prognosticate where the field is heading.
Subject(s)
Other Subject(s)
Genre(s)
ISBN
9781636392295 electronic
9781636392301 hardcover
9781636392288 paperback
Note
Part of: Synthesis digital library of engineering and computer science.
Bibliography Note
Includes bibliographical references.
Other Forms
Also available in print.
Technical Details
Mode of access: World Wide Web.
System requirements: Adobe Acrobat Reader.
Indexed By
Compendex
INSPEC
Google scholar
Google book search
View MARC record | catkey: 37114346