Title Speaker Name Introduction to natural

Agenda Introduction to natural language processing with TensorFlow
Representing text as Tensors
Represent words with embeddings
Capture patterns with recurrent neural networks
Generate text with recurrent networks<br>

Introduction to natural language processing with TensorFlow<br>

Introduction to natural language processing with TensorFlow In this presentation, we will explore different neural network architectures for dealing with natural language text.<br>

Natural Language Tasks There are several NLP tasks that we can solve using neural networks: Text Classification is used when we need to classify a text fragment into one of several predefined classes. Examples include e-mail spam detection, news categorization, assigning a support request to a category, and more.
Intent Classification is one specific case of text classification, where we want to map an input utterance in the conversational AI system into one of the intents that represent the actual meaning of the phrase, or intent of the user.
Sentiment Analysis is a regression task, where we want to understand the degree of positivity of a given piece of text. We may want to label text in a dataset from most negative (-1) to most positive (+1), and train a model that will output a number representing the positivity of the input text.
Named Entity Recognition (NER) is the task of extracting entities from text, such as dates, addresses, people names, etc. Together with intent classification, NER is often used in dialog systems to extract parameters from the user's utterance.
A similar task of Keyword Extraction can be used to find the most meaningful words inside a text, which can then be used as tags.
Text Summarization extracts the most meaningful pieces of text, giving the user a compressed version of the original text.
Question Answering is the task of extracting an answer from a piece of text. This model takes a text fragment and a question as input, and finds the exact place within the text that contains the answer. For example, the text "John is a 22 year old student who loves to use Microsoft Learn", and the question How old is John should provide us with the answer 22.<br>

Essential Facts<br>

Use embedding layer before fully-connected classifier layer Reduces the dimensionality of the input vector to neural classifier<br>

Use character-level LSTM To generate words with specific meaning, like funny ones.<br>

LSTM network architecture Allows explicit state management with forgetting and state triggering<br>

Question 1 Suppose your text corpus contains 80000 different words. Which of the below would you complete to reduce the dimensionality of the input vector to neural classifier? Randomly select 10% of the words and ignore the rest.
Use convolutional layer before fully-connected classifier layer
Use embedding layer before fully-connected classifier layer
Select 10% of most frequently used words and ignore the rest<br>

Question 1 Suppose your text corpus contains 80000 different words. Which of the below would you complete to reduce the dimensionality of the input vector to neural classifier? Randomly select 10% of the words and ignore the rest.
Use convolutional layer before fully-connected classifier layer
Use embedding layer before fully-connected classifier layer
Select 10% of most frequently used words and ignore the rest<br>

Question 2 We want to train a neural network to generate new funny words for a children's book. Which architecture can we use? Word-level LSTM
Character-level LSTM
Word-level RNN
Character-level perceptron<br>

Question 2 We want to train a neural network to generate new funny words for a children's book. Which architecture can we use? Word-level LSTM
Character-level LSTM
Word-level RNN
Character-level perceptron<br>

Question 3 Recurrent neural network is called recurrent, because: A network is applied for each input element and output from the previous application is passed to the next one
It is trained by a recurrent process
It consists of layers which include other subnetworks<br>

Question 3 Recurrent neural network is called recurrent, because: A network is applied for each input element and output from the previous application is passed to the next one
It is trained by a recurrent process
It consists of layers which include other subnetworks<br>

Question 4 What is the main idea behind LSTM network architecture? Fixed number of LSTM blocks for the whole dataset
It contains many layers of recurrent neural networks
Explicit state management with forgetting and state triggering<br>

Question 4 What is the main idea behind LSTM network architecture? Fixed number of LSTM blocks for the whole dataset
It contains many layers of recurrent neural networks
Explicit state management with forgetting and state triggering<br>

Question 5 What is the main idea of attention? Attention assigns a weight coefficient to each word in the vocabulary to show how important it is
Attention is a network layer that uses attention matrix to see how much input states from each step affect the final result.
Attention builds global correlation matrix between all words in vocabulary, showing their co-occurrence<br>

Question 5 What is the main idea of attention? Attention assigns a weight coefficient to each word in the vocabulary to show how important it is
Attention is a network layer that uses attention matrix to see how much input states from each step affect the final result.
Attention builds global correlation matrix between all words in vocabulary, showing their co-occurrence<br>

Summary<br>

Summary We have covered all the basics of Natural Language Processing such as: text representation, traditional recurrent network models, and near state-of-the-art models with attention.<br>

Next Steps Practice your knowledge by trying these Learn modules: Please tell us how you liked this workshop by filling out this survey:
https://aka.ms/workshopomatic-feedback There are other Learn Modules for TensorFlow that are grouped in the TensorFlow fundamentals Learning Path<br>