SUPARN PATRA
Schools
Exams
Paths
Blog
Quizzes
Tools
Playground
About
Home
/
Quizzes
/
AI / ML Path
/
NLP and LLMs
AI / ML Path
NLP and LLMs
10 questions
Question 1 of 10
Score:
0
/ 0
What is the purpose of tokenisation in NLP?
A
To remove stop words from text
B
To split text into units (tokens) and map each to a numeric ID so models can process it
C
To convert tokens to lowercase
D
To measure the similarity between sentences
← Back to lesson notes