Amazon Web Services Bloom Z 176b Few Shot And Zero Shot Learning

Bloom Z 176b Few Shot And Zero Shot Learning

data-scienceinferencearchivedamazon-sagemaker-examplesreinforcement-learningmachine-learningawsexamplesdeep-learningsagemakerjupyter-notebooktrainingmlops

alph-notebooks/amazon-sagemaker-examples / bloom-z-176b-few-shot-and-zero-shot-learning.ipynb

Export

Run Notebooks

Contents

No cells yet

Add cells to see them here

SageMaker JumpStart Foundation Models - BloomZ 176B FP16 Few-shot and Zero-shot learning

This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook.

Welcome to Amazon SageMaker JumpStart! You can use Sagemaker JumpStart to solve many Machine Learning tasks through one-click in SageMaker Studio, or through SageMaker Python SDK.

In this demo notebook, we demonstrate how to use the SageMaker Python SDK for deploying Foundation Models as an endpoint and use them for various NLP tasks. The Foundation models perform Text Generation. It takes a text string as input and predicts a sequence of next few words. These models can, for example, fill in incomplete text or paraphrase.

Here, we show how to use the state-of-the-art instruction-tuned BloomZ 176B model from Hugging Face for Text Generation in the following tasks. You can use BloomZ 176B model model with few shot learning and zero shot learning for many NLP tasks, without fine-tuning the model.

The few shot leanring allows a machine learning model to make predictions with only a couple of examples. There is no need to train a new model as models like BloomZ 176B model have a significant amount of parameters such that they can easily adapt to many contexts without being re-trained. Below are tasks that text generation can solve using few shot learning method.

Few-shot learning
- Text summarization
- Code generation
- Name entity recognition (NER)
- Question answering
- Grammar and spelling correction
- Product description and generalization
- Sentence / sentiment classification
- Chatbot and conversational AI
- Tweet generation
- Machine translation
- Intent classification
Zero-shot learning
- Multilingual text / sentiment classification
- Multilingual question and answering
- Code generation
- Paragraph rephrase
- Summarization
- Common sense reasoning / natural language inference
- Question and answering
- Sentence / sentiment classification
- Imaginary article generation based on title
- Summarize a title based on a article

Note. Deploying this model requires a p4de.24xlarge instance and the deployment usually takes about 1 hour.

If you don't have quota for such instance, please go to AWS console -> Service Quotas -> AWS services to request it.

Set Up
Select a model
Retrieve Artifacts & Deploy an Endpoint
Query endpoint and parse response
Advanced features: How to use varisous parameters to control the generated text
Advanced features: How to use prompts engineering to solve different tasks
Clean up the endpoint

Note: This notebook was tested on ml.t3.medium instance in Amazon SageMaker Studio with Python 3 (Data Science) kernel and in Amazon SageMaker Notebook instance with conda_python3 kernel.

1. Set Up

Before executing the notebook, there are some initial steps required for set up. This notebook requires ipywidgets.

[ ]

Permissions and environment variables

To host on Amazon SageMaker, we need to set up and authenticate the use of AWS services. Here, we use the execution role associated with the current notebook as the AWS account role with SageMaker access.

[5]

2. Select a pre-trained model

You can continue with the default model, or can choose a different model from the dropdown generated upon running the next cell. A complete list of SageMaker pre-trained models can also be accessed at Sagemaker pre-trained Models.

[11]

[Optional] Select a different Sagemaker pre-trained model. Here, we download the model_manifest file from the Built-In Algorithms s3 bucket, filter-out all the Text Generation models and select a model for inference.

The notebook in the following sections uses BloomZ 176B as an example. You are welcome to try BloomZ 7b1, BloomZ 3b, and many others yourself. Please modify model_id based on the dropdown list shown as below.

[12]

Choose a model for Inference

[13]

A Jupyter Widget

[14]

3. Retrieve Artifacts & Deploy an Endpoint

Using SageMaker, we can perform inference on the pre-trained model, even without fine-tuning it first on a new dataset. We start by retrieving the deploy_image_uri, deploy_source_uri, and model_uri for the pre-trained model. To host the pre-trained model, we create an instance of sagemaker.model.Model and deploy it. This may take a few minutes.

[8]

'huggingface-textgeneration1-bloomz-176b-fp16'

Now we deploy the model using simiplified JumpStart SDK with following few lines of code.

Note. Deploying this model requires a p4de.24xlarge instance and the deployment usually takes about 1 hour. If you don't have quota for such instance, please go to AWS console -> Service Quotas -> AWS services to request it.

[18]

---------------------------------------------------------------------------------------------------------!Elastic time: 3498.1374440193176

4. Query endpoint and parse response

Input to the endpoint is any string of text formatted as json and encoded in utf-8 format. Output of the endpoint is a json with generated text.

[19]

Below, we put in some example input text. You can put in any text and the model predicts next words in the sequence. Longer sequences of text can be generated by calling the model repeatedly.

[24]

Inference:
input text: I will
generated text: ['"I will" is a future tense, so it is used to talk about what will happen in the']

Inference:
input text: The movie is so funny
generated text: ['"The movie is so funny"']

5. Advanced features: How to use various advanced parameters to control the generated text

This model also supports many advanced parameters while performing inference. They include:

max_length: Model generates text until the output length (which includes the input context length) reaches max_length. If specified, it must be a positive integer.
num_return_sequences: Number of output sequences returned. If specified, it must be a positive integer.
num_beams: Number of beams used in the greedy search. If specified, it must be integer greater than or equal to num_return_sequences.
no_repeat_ngram_size: Model ensures that a sequence of words of no_repeat_ngram_size is not repeated in the output sequence. If specified, it must be a positive integer greater than 1.
temperature: Controls the randomness in the output. Higher temperature results in output sequence with low-probability words and lower temperature results in output sequence with high-probability words. If temperature -> 0, it results in greedy decoding. If specified, it must be a positive float.
early_stopping: If True, text generation is finished when all beam hypotheses reach the end of stence token. If specified, it must be boolean.
do_sample: If True, sample the next word as per the likelyhood. If specified, it must be boolean.
top_k: In each step of text generation, sample from only the top_k most likely words. If specified, it must be a positive integer.
top_p: In each step of text generation, sample from the smallest possible set of words with cumulative probability top_p. If specified, it must be a float between 0 and 1.
seed: Fix the randomized state for reproducibility. If specified, it must be an integer.

We may specify any subset of the parameters mentioned above while invoking an endpoint. Next, we show an example of how to invoke endpoint with these arguments

[31]

['How to make a pasta? boil water in a pot, then add the pasta.', 'How to make a pasta? boil water in a pot, add salt and pasta.']

6. Advanced features: How to use prompts engineering to solve different few shot and zero shot learning NLP tasks

Note . The notebook in the following sections uses BloomZ 176B as an example. You are welcome to try BloomZ 7b1, BloomZ 3b and many others yourself. Please modify model_id in above section.

Let's first try few shot learning tasks

6.1. Summarization

Define the text article you want to summarize.

[66]

[67]

Number of return sequences are set as 2

For prompt: '[Original]: Amazon scientists, in collaboration with researchers from the University of Sheffield, are making a large-scale fact extraction and verification dataset publicly available for the first time. The dataset, comprising more than 185,000 evidence-backed claims, is being made available to hopefully catalyze research and development that addresses the problems of fact extraction and verification in software applications or cloud-based services that perform automatic information extraction.
[Summary]: Amazon and University researchers make fact extraction and verification dataset publicly available.
###
[Original]: Prime members in the U.S. can get even more delivered to their door with a Prime membership. Members can now enjoy one year of Grubhub+ valued at $9.99 per month for free—at no added cost to their Prime membership. To activate this deal, visit amazon.com/grubhub. This new offer includes unlimited, $0 food delivery fees on orders over $12 as well as exclusive perks for Grubhub+ members and rewards like free food and order discounts. Plus, diners can “eat good while doing good” by opting into Grubhub’s Donate the Change program, a donation-matching initiative that raised more than $25 million in 2021 alone, benefiting more than 20 charitable organizations across the country.
[Summary]: Prime members in the U.S. can enjoy one year of Grubhub+ for free, with no food-delivery fees on eligible orders.
###
[Original]: {text}
[Summary]:'

The 2 results are:

Result 0: [Original]: Amazon scientists, in collaboration with researchers from the University of Sheffield, are making a large-scale fact extraction and verification dataset publicly available for the first time. The dataset, comprising more than 185,000 evidence-backed claims, is being made available to hopefully catalyze research and development that addresses the problems of fact extraction and verification in software applications or cloud-based services that perform automatic information extraction.
[Summary]: Amazon and University researchers make fact extraction and verification dataset publicly available.
###
[Original]: Prime members in the U.S. can get even more delivered to their door with a Prime membership. Members can now enjoy one year of Grubhub+ valued at $9.99 per month for free—at no added cost to their Prime membership. To activate this deal, visit amazon.com/grubhub. This new offer includes unlimited, $0 food delivery fees on orders over $12 as well as exclusive perks for Grubhub+ members and rewards like free food and order discounts. Plus, diners can “eat good while doing good” by opting into Grubhub’s Donate the Change program, a donation-matching initiative that raised more than $25 million in 2021 alone, benefiting more than 20 charitable organizations across the country.
[Summary]: Prime members in the U.S. can enjoy one year of Grubhub+ for free, with no food-delivery fees on eligible orders.
###
[Original]: Amazon Comprehend uses natural language processing (NLP) to extract insights about the content of documents. It develops insights by recognizing the entities, key phrases, language, sentiments, and other common elements in a document. Use Amazon Comprehend to create new products based on understanding the structure of documents. For example, using Amazon Comprehend you can search social networking feeds for mentions of products or scan an entire document repository for key phrases.
You can access Amazon Comprehend document analysis capabilities using the Amazon Comprehend console or using the Amazon Comprehend APIs. You can run real-time analysis for small workloads or you can start asynchronous analysis jobs for large document sets. You can use the pre-trained models that Amazon Comprehend provides, or you can train your own custom models for classification and entity recognition.
[Summary]: Amazon Comprehend makes it easy to extract insights from documents.

Result 1: [Original]: Amazon scientists, in collaboration with researchers from the University of Sheffield, are making a large-scale fact extraction and verification dataset publicly available for the first time. The dataset, comprising more than 185,000 evidence-backed claims, is being made available to hopefully catalyze research and development that addresses the problems of fact extraction and verification in software applications or cloud-based services that perform automatic information extraction.
[Summary]: Amazon and University researchers make fact extraction and verification dataset publicly available.
###
[Original]: Prime members in the U.S. can get even more delivered to their door with a Prime membership. Members can now enjoy one year of Grubhub+ valued at $9.99 per month for free—at no added cost to their Prime membership. To activate this deal, visit amazon.com/grubhub. This new offer includes unlimited, $0 food delivery fees on orders over $12 as well as exclusive perks for Grubhub+ members and rewards like free food and order discounts. Plus, diners can “eat good while doing good” by opting into Grubhub’s Donate the Change program, a donation-matching initiative that raised more than $25 million in 2021 alone, benefiting more than 20 charitable organizations across the country.
[Summary]: Prime members in the U.S. can enjoy one year of Grubhub+ for free, with no food-delivery fees on eligible orders.
###
[Original]: Amazon Comprehend uses natural language processing (NLP) to extract insights about the content of documents. It develops insights by recognizing the entities, key phrases, language, sentiments, and other common elements in a document. Use Amazon Comprehend to create new products based on understanding the structure of documents. For example, using Amazon Comprehend you can search social networking feeds for mentions of products or scan an entire document repository for key phrases.
You can access Amazon Comprehend document analysis capabilities using the Amazon Comprehend console or using the Amazon Comprehend APIs. You can run real-time analysis for small workloads or you can start asynchronous analysis jobs for large document sets. You can use the pre-trained models that Amazon Comprehend provides, or you can train your own custom models for classification and entity recognition.
[Summary]: Amazon Comprehend is a document analysis service that uses natural language processing (NLP) to extract insights from documents.

6.2. Code generation

[68]

[69]

 For prompt: 'description: a orange button that says stop
    code: <button style=color:white; background-color:orange;>Stop</button>
    ###
    description: a blue box that contains yellow circles with red borders
    code: <div style=background-color: blue; padding: 20px;><div style=background-color: yellow; border: 5px solid red; border-radius: 50%; padding: 20px; width: 100px; height: 100px;>
    ###
    description: {description}
    code:'

 The 2 results are:

 The reasoning result 0 is: 'description: a orange button that says stop
    code: <button style=color:white; background-color:orange;>Stop</button>
    ###
    description: a blue box that contains yellow circles with red borders
    code: <div style=background-color: blue; padding: 20px;><div style=background-color: yellow; border: 5px solid red; border-radius: 50%; padding: 20px; width: 100px; height: 100px;>
    ###
    description: a Headline saying Welcome to AI
    code: <h1>Welcome to AI</h1>'

 The reasoning result 1 is: 'description: a orange button that says stop
    code: <button style=color:white; background-color:orange;>Stop</button>
    ###
    description: a blue box that contains yellow circles with red borders
    code: <div style=background-color: blue; padding: 20px;><div style=background-color: yellow; border: 5px solid red; border-radius: 50%; padding: 20px; width: 100px; height: 100px;>
    ###
    description: a Headline saying Welcome to AI
    code: <h1 style=color:white;>Welcome to AI</h1>'

How about SQL code?

[70]

[71]

 For prompt: 'Question: Fetch the companies that have less than five people in it.
    Answer: SELECT COMPANY, COUNT(EMPLOYEE_ID) FROM Employee GROUP BY COMPANY HAVING COUNT(EMPLOYEE_ID) < 5;
    ###
    Question: Show all companies along with the number of employees in each department
    Answer: SELECT COMPANY, COUNT(COMPANY) FROM Employee GROUP BY COMPANY;
    ###
    Question: Show the last record of the Employee table
    Answer: SELECT * FROM Employee ORDER BY LAST_NAME DESC LIMIT 1;
    ###
    Question: {question};
    Answer:'

 The 2 results are:

 The reasoning result 0 is: 'Question: Fetch the companies that have less than five people in it.
    Answer: SELECT COMPANY, COUNT(EMPLOYEE_ID) FROM Employee GROUP BY COMPANY HAVING COUNT(EMPLOYEE_ID) < 5;
    ###
    Question: Show all companies along with the number of employees in each department
    Answer: SELECT COMPANY, COUNT(COMPANY) FROM Employee GROUP BY COMPANY;
    ###
    Question: Show the last record of the Employee table
    Answer: SELECT * FROM Employee ORDER BY LAST_NAME DESC LIMIT 1;
    ###
    Question: Fetch three employees from the Employee table;
    Answer: SELECT * FROM Employee LIMIT 3;'

 The reasoning result 1 is: 'Question: Fetch the companies that have less than five people in it.
    Answer: SELECT COMPANY, COUNT(EMPLOYEE_ID) FROM Employee GROUP BY COMPANY HAVING COUNT(EMPLOYEE_ID) < 5;
    ###
    Question: Show all companies along with the number of employees in each department
    Answer: SELECT COMPANY, COUNT(COMPANY) FROM Employee GROUP BY COMPANY;
    ###
    Question: Show the last record of the Employee table
    Answer: SELECT * FROM Employee ORDER BY LAST_NAME DESC LIMIT 1;
    ###
    Question: Fetch three employees from the Employee table;
    Answer: SELECT * FROM Employee LIMIT 3;
    ###'

6.3. Name entity recognition (NER)

[72]

[73]

 For prompt: '[Text]: Fred is a serial entrepreneur. Co-founder and CEO of Platform.sh, he previously co-founded Commerce Guys, a leading Drupal ecommerce provider. His mission is to guarantee that as we continue on an ambitious journey to profoundly transform how cloud computing is used and perceived, we keep our feet well on the ground continuing the rapid growth we have enjoyed up until now. 
    [Name]: Fred
    [Position]: Co-founder and CEO
    [Company]: Platform.sh
    ###
    [Text]: Microsoft (the word being a portmanteau of "microcomputer software") was founded by Bill Gates on April 4, 1975, to develop and sell BASIC interpreters for the Altair 8800. Steve Ballmer replaced Gates as CEO in 2000, and later envisioned a "devices and services" strategy.
    [Name]:  Steve Ballmer
    [Position]: CEO
    [Company]: Microsoft
    ###
    [Text]: Franck Riboud was born on 7 November 1955 in Lyon. He is the son of Antoine Riboud, the previous CEO, who transformed the former European glassmaker BSN Group into a leading player in the food industry. He is the CEO at Danone.
    [Name]:  Franck Riboud
    [Position]: CEO
    [Company]: Danone
    ###
    [Text]: {sentence}
    '

 The 2 results are:

 The reasoning result 0 is: '[Text]: Fred is a serial entrepreneur. Co-founder and CEO of Platform.sh, he previously co-founded Commerce Guys, a leading Drupal ecommerce provider. His mission is to guarantee that as we continue on an ambitious journey to profoundly transform how cloud computing is used and perceived, we keep our feet well on the ground continuing the rapid growth we have enjoyed up until now. 
    [Name]: Fred
    [Position]: Co-founder and CEO
    [Company]: Platform.sh
    ###
    [Text]: Microsoft (the word being a portmanteau of "microcomputer software") was founded by Bill Gates on April 4, 1975, to develop and sell BASIC interpreters for the Altair 8800. Steve Ballmer replaced Gates as CEO in 2000, and later envisioned a "devices and services" strategy.
    [Name]:  Steve Ballmer
    [Position]: CEO
    [Company]: Microsoft
    ###
    [Text]: Franck Riboud was born on 7 November 1955 in Lyon. He is the son of Antoine Riboud, the previous CEO, who transformed the former European glassmaker BSN Group into a leading player in the food industry. He is the CEO at Danone.
    [Name]:  Franck Riboud
    [Position]: CEO
    [Company]: Danone
    ###
    [Text]: David Melvin is an investment and financial services professional at CITIC CLSA with over 30 years’ experience in investment banking and private equity. He is currently a Senior Adviser of CITIC CLSA.
     [Name]: David Melvin
    [Position]: Senior Adviser
    [Company]: CITIC CLSA'

 The reasoning result 1 is: '[Text]: Fred is a serial entrepreneur. Co-founder and CEO of Platform.sh, he previously co-founded Commerce Guys, a leading Drupal ecommerce provider. His mission is to guarantee that as we continue on an ambitious journey to profoundly transform how cloud computing is used and perceived, we keep our feet well on the ground continuing the rapid growth we have enjoyed up until now. 
    [Name]: Fred
    [Position]: Co-founder and CEO
    [Company]: Platform.sh
    ###
    [Text]: Microsoft (the word being a portmanteau of "microcomputer software") was founded by Bill Gates on April 4, 1975, to develop and sell BASIC interpreters for the Altair 8800. Steve Ballmer replaced Gates as CEO in 2000, and later envisioned a "devices and services" strategy.
    [Name]:  Steve Ballmer
    [Position]: CEO
    [Company]: Microsoft
    ###
    [Text]: Franck Riboud was born on 7 November 1955 in Lyon. He is the son of Antoine Riboud, the previous CEO, who transformed the former European glassmaker BSN Group into a leading player in the food industry. He is the CEO at Danone.
    [Name]:  Franck Riboud
    [Position]: CEO
    [Company]: Danone
    ###
    [Text]: David Melvin is an investment and financial services professional at CITIC CLSA with over 30 years’ experience in investment banking and private equity. He is currently a Senior Adviser of CITIC CLSA.
     [Name]: David Melvin
    [Position]: Senior Adviser
    [Company]: CITIC CLSA'

6.4. Question answering

[74]

[75]

For prompt: 'Context: NLP Cloud was founded in 2021 when the team realized there was no easy way to reliably leverage Natural Language Processing in production.
Question: When was NLP Cloud founded?
Answer: 2021
###
Context: NLP Cloud developed their API by mid-2020 and they added many pre-trained open-source models since then.
Question: What did NLP Cloud develop?
Answer: API
###
Context: All plans can be stopped anytime. You only pay for the time you used the service. In case of a downgrade, you will get a discount on your next invoice.
Question: When can plans be stopped?
Answer: Anytime
###
Context: The main challenge with GPT-J is memory consumption. Using a GPU plan is recommended.
Question: {question}
Answer:'

The 2 results are:

The reasoning result 0 is: 'Context: NLP Cloud was founded in 2021 when the team realized there was no easy way to reliably leverage Natural Language Processing in production.
Question: When was NLP Cloud founded?
Answer: 2021
###
Context: NLP Cloud developed their API by mid-2020 and they added many pre-trained open-source models since then.
Question: What did NLP Cloud develop?
Answer: API
###
Context: All plans can be stopped anytime. You only pay for the time you used the service. In case of a downgrade, you will get a discount on your next invoice.
Question: When can plans be stopped?
Answer: Anytime
###
Context: The main challenge with GPT-J is memory consumption. Using a GPU plan is recommended.
Question: Which plan is recommended for GPT-J?
Answer: GPU'

The reasoning result 1 is: 'Context: NLP Cloud was founded in 2021 when the team realized there was no easy way to reliably leverage Natural Language Processing in production.
Question: When was NLP Cloud founded?
Answer: 2021
###
Context: NLP Cloud developed their API by mid-2020 and they added many pre-trained open-source models since then.
Question: What did NLP Cloud develop?
Answer: API
###
Context: All plans can be stopped anytime. You only pay for the time you used the service. In case of a downgrade, you will get a discount on your next invoice.
Question: When can plans be stopped?
Answer: Anytime
###
Context: The main challenge with GPT-J is memory consumption. Using a GPU plan is recommended.
Question: Which plan is recommended for GPT-J?
Answer: GPU'

6.5. Grammar and spelling correction

[76]

[77]

 For prompt: 'I love goin to the beach.
    Correction: I love going to the beach.
    ###
    Let me hav it!
    Correction: Let me have it!
    ###
    It have too many drawbacks.
    Correction: It has too many drawbacks.
    ###
    {sentence}
    Correction:'

 The 2 results are:

 The reasoning result 0 is: 'I love goin to the beach.
    Correction: I love going to the beach.
    ###
    Let me hav it!
    Correction: Let me have it!
    ###
    It have too many drawbacks.
    Correction: It has too many drawbacks.
    ###
    I do not wan to go
    Correction: I don't want to go
    ###'

 The reasoning result 1 is: 'I love goin to the beach.
    Correction: I love going to the beach.
    ###
    Let me hav it!
    Correction: Let me have it!
    ###
    It have too many drawbacks.
    Correction: It has too many drawbacks.
    ###
    I do not wan to go
    Correction: I don't want to go
    ###'

6.6. Product description and generalization

[78]

[79]

 For prompt: 'Generate a product description out of keywords.

    Keywords: shoes, women, $59
    Sentence: Beautiful shoes for women at the price of $59.
    ###
    Keywords: trousers, men, $69
    Sentence: Modern trousers for men, for $69 only.
    ###
    Keywords: gloves, winter, $19
    Sentence: Amazingly hot gloves for cold winters, at $19.
    ###
    Keywords: {sentence}
    Sentence:'

 The 2 results are:

 The reasoning result 0 is: 'Generate a product description out of keywords.

    Keywords: shoes, women, $59
    Sentence: Beautiful shoes for women at the price of $59.
    ###
    Keywords: trousers, men, $69
    Sentence: Modern trousers for men, for $69 only.
    ###
    Keywords: gloves, winter, $19
    Sentence: Amazingly hot gloves for cold winters, at $19.
    ###
    Keywords: t-shirt, men, $39
    Sentence: Cool t-shirt for men, at $39.'

 The reasoning result 1 is: 'Generate a product description out of keywords.

    Keywords: shoes, women, $59
    Sentence: Beautiful shoes for women at the price of $59.
    ###
    Keywords: trousers, men, $69
    Sentence: Modern trousers for men, for $69 only.
    ###
    Keywords: gloves, winter, $19
    Sentence: Amazingly hot gloves for cold winters, at $19.
    ###
    Keywords: t-shirt, men, $39
    Sentence: Cool t-shirt for men, for $39 only.'

6.7. Sentence / Sentiment Classification

Define the sentence you want to classifiy and the corresponded options.

[80]

[81]

 For prompt: 'Message: When the spaceship landed on Mars, the whole humanity was excited
    Topic: space
    ###
    Message: I love playing tennis and golf. I'm practicing twice a week.
    Topic: sport
    ###
    Message: Managing a team of sales people is a tough but rewarding job.
    Topic: business
    ###
    Message: I am trying to cook chicken with tomatoes.
    Topic:'

 The 2 results are:

 The reasoning result 0 is: 'Message: When the spaceship landed on Mars, the whole humanity was excited
    Topic: space
    ###
    Message: I love playing tennis and golf. I'm practicing twice a week.
    Topic: sport
    ###
    Message: Managing a team of sales people is a tough but rewarding job.
    Topic: business
    ###
    Message: I am trying to cook chicken with tomatoes.
    Topic: food'

 The reasoning result 1 is: 'Message: When the spaceship landed on Mars, the whole humanity was excited
    Topic: space
    ###
    Message: I love playing tennis and golf. I'm practicing twice a week.
    Topic: sport
    ###
    Message: Managing a team of sales people is a tough but rewarding job.
    Topic: business
    ###
    Message: I am trying to cook chicken with tomatoes.
    Topic: food'

6.8. Chatbot and Conversational AI

Define the sentence and the language you want to translate the sentence to.

[82]

[83]

 For prompt: 'This is a discussion between a [human] and a [robot]. 
    The [robot] is very nice and empathetic.

    [human]: Hello nice to meet you.
    [robot]: Nice to meet you too.
    ###
    [human]: How is it going today?
    [robot]: Not so bad, thank you! How about you?
    ###
    [human]: I am ok, but I am a bit sad...
    [robot]: Oh? Why that?
    ###
    [human]: I caught flu
    [robot]:'

 The 2 results are:

 The reasoning result 0 is: 'This is a discussion between a [human] and a [robot]. 
    The [robot] is very nice and empathetic.

    [human]: Hello nice to meet you.
    [robot]: Nice to meet you too.
    ###
    [human]: How is it going today?
    [robot]: Not so bad, thank you! How about you?
    ###
    [human]: I am ok, but I am a bit sad...
    [robot]: Oh? Why that?
    ###
    [human]: I caught flu
    [robot]: Oh no, I am sorry to hear that.'

 The reasoning result 1 is: 'This is a discussion between a [human] and a [robot]. 
    The [robot] is very nice and empathetic.

    [human]: Hello nice to meet you.
    [robot]: Nice to meet you too.
    ###
    [human]: How is it going today?
    [robot]: Not so bad, thank you! How about you?
    ###
    [human]: I am ok, but I am a bit sad...
    [robot]: Oh? Why that?
    ###
    [human]: I caught flu
    [robot]: Oh no, I am sorry to hear that!'

6.9. Tweet generation

Define the sentence, pronoun, and options you want to reason.

[84]

[85]

 For prompt: 'keyword: markets
    tweet: Take feedback from nature and markets, not from people
    ###
    keyword: children
    tweet: Maybe we die so we can come back as children.
    ###
    keyword: startups
    tweet: Startups should not worry about how to put out fires, they should worry about how to start them.
    ###
    keyword: nlp
    tweet:'

 The 2 results are:

 The reasoning result 0 is: 'keyword: markets
    tweet: Take feedback from nature and markets, not from people
    ###
    keyword: children
    tweet: Maybe we die so we can come back as children.
    ###
    keyword: startups
    tweet: Startups should not worry about how to put out fires, they should worry about how to start them.
    ###
    keyword: nlp
    tweet: NLP is a tool, not a goal.'

 The reasoning result 1 is: 'keyword: markets
    tweet: Take feedback from nature and markets, not from people
    ###
    keyword: children
    tweet: Maybe we die so we can come back as children.
    ###
    keyword: startups
    tweet: Startups should not worry about how to put out fires, they should worry about how to start them.
    ###
    keyword: nlp
    tweet: NLP is a tool, not a goal, not a religion.'

6.10. Machine translation

[86]

[87]

 For prompt: 'Hugging Face a révolutionné le NLP.
    Translation: Hugging Face revolutionized NLP.
    ###
    Cela est incroyable!
    Translation: This is unbelievable!
    ###
    Désolé je ne peux pas.
    Translation: Sorry but I cannot.
    ###
    NLP Cloud permet de deployer le NLP en production facilement.
    Translation:'

 The 2 summarized results are:

 The reasoning result 0 is: 'Hugging Face a révolutionné le NLP.
    Translation: Hugging Face revolutionized NLP.
    ###
    Cela est incroyable!
    Translation: This is unbelievable!
    ###
    Désolé je ne peux pas.
    Translation: Sorry but I cannot.
    ###
    NLP Cloud permet de deployer le NLP en production facilement.
    Translation: NLP Cloud makes it easy to deploy NLP in production.'

 The reasoning result 1 is: 'Hugging Face a révolutionné le NLP.
    Translation: Hugging Face revolutionized NLP.
    ###
    Cela est incroyable!
    Translation: This is unbelievable!
    ###
    Désolé je ne peux pas.
    Translation: Sorry but I cannot.
    ###
    NLP Cloud permet de deployer le NLP en production facilement.
    Translation: NLP Cloud makes it easy to deploy NLP in production.
    ###'

6.11. Intent classification

[88]

[89]

 For prompt: 'I want to start coding tomorrow because it seems to be so fun!
    Intent: start coding
    ###
    Show me the last pictures you have please.
    Intent: show pictures
    ###
    Search all these files as fast as possible.
    Intent: search files
    ###
    Can you please teach me Chinese next week?
    Intent:'

 The 2 summarized results are:

 The reasoning result 0 is: 'I want to start coding tomorrow because it seems to be so fun!
    Intent: start coding
    ###
    Show me the last pictures you have please.
    Intent: show pictures
    ###
    Search all these files as fast as possible.
    Intent: search files
    ###
    Can you please teach me Chinese next week?
    Intent: teach me chinese
    ###'

 The reasoning result 1 is: 'I want to start coding tomorrow because it seems to be so fun!
    Intent: start coding
    ###
    Show me the last pictures you have please.
    Intent: show pictures
    ###
    Search all these files as fast as possible.
    Intent: search files
    ###
    Can you please teach me Chinese next week?
    Intent: teach me chinese
    ###'

Next, let's try zero shot learning tasks

6.13. Multilingual text / sentiment classification (Chinese to English)

[91]

[94]

For input with prompt: 一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。 Would you rate the previous review as positive, neutral or negative?
 The 2 results are:

 The reasoning result 0 is: '一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。 Would you rate the previous review as positive, neutral or negative? positive'

 The reasoning result 1 is: '一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。 Would you rate the previous review as positive, neutral or negative? positive'

6.14. Multilingual question and answering (English to Chinese)

[95]

[99]

For input with prompt: Explain to me in Traditional Chinese  what is the backpropagation.
 The 2 results are:

 The reasoning result 0 is: 'Explain to me in Traditional Chinese  what is the backpropagation. 梯度反向传播算法是一种机器学习算法，用于训练人工神经网络。该算法使用梯度下降来最小化损失函数。'

 The reasoning result 1 is: 'Explain to me in Traditional Chinese  what is the backpropagation. 遞迴神經網絡的反向傳播算法，是一種用於訓練遞迴神經網絡的算法。'

6.15. Code generation

[100]

[101]

[102]

For input with prompt: Write a binary search tree with O(log(n)) computational complexity. 

def binary_search(a, x):
    low = 0
    high = len(a) - 1
 For prompt: 'Write a {text} with O(log(n)) computational complexity. 
{code_start}'

 The 2 results are:

 The reasoning result 0 is: 'Write a binary search tree with O(log(n)) computational complexity. 

def binary_search(a, x):
    low = 0
    high = len(a) - 1
    while low <= high:
        mid = (low + high) // 2
        if a[mid] < x:
            low = mid + 1
        elif a[mid] > x:
            high = mid - 1
        else:
            return mid
    return -1'

 The reasoning result 1 is: 'Write a binary search tree with O(log(n)) computational complexity. 

def binary_search(a, x):
    low = 0
    high = len(a) - 1
    while low <= high:
        mid = (low + high) // 2
        if a[mid] < x:
            low = mid + 1
        elif a[mid] > x:
            high = mid - 1
        else:
            return mid
    return low'

6.16. Paragraph rephrase

[103]

[105]

For prompt: '\ Amazon Web Services (AWS) has announced nine major new updates for its cloud-based machine learning platform, SageMaker.
SageMaker aims to provide a machine learning service which can be used to build, train, and deploy ML models for virtually any use case.
During this year’s re:Invent conference, AWS made several announcements to further improve SageMaker’s capabilities.

How would you rephrase that briefly using English?'

The 2 results are:

The reasoning result 0 is: '\ Amazon Web Services (AWS) has announced nine major new updates for its cloud-based machine learning platform, SageMaker.
SageMaker aims to provide a machine learning service which can be used to build, train, and deploy ML models for virtually any use case.
During this year’s re:Invent conference, AWS made several announcements to further improve SageMaker’s capabilities.

How would you rephrase that briefly using English? Amazon SageMaker gets a major update'

The reasoning result 1 is: '\ Amazon Web Services (AWS) has announced nine major new updates for its cloud-based machine learning platform, SageMaker.
SageMaker aims to provide a machine learning service which can be used to build, train, and deploy ML models for virtually any use case.
During this year’s re:Invent conference, AWS made several announcements to further improve SageMaker’s capabilities.

How would you rephrase that briefly using English? Amazon SageMaker gets a major update'

For prompt: '"Amazon Web Services (AWS) has announced nine major new updates for its cloud-based machine learning platform, SageMaker.
SageMaker aims to provide a machine learning service which can be used to build, train, and deploy ML models for virtually any use case.
During this year’s re:Invent conference, AWS made several announcements to further improve SageMaker’s capabilities.
The above sentence is very complicated. Please provide me a simplified synonymous version consisting of multiple sentences:'

The 2 results are:

The reasoning result 0 is: '"Amazon Web Services (AWS) has announced nine major new updates for its cloud-based machine learning platform, SageMaker.
SageMaker aims to provide a machine learning service which can be used to build, train, and deploy ML models for virtually any use case.
During this year’s re:Invent conference, AWS made several announcements to further improve SageMaker’s capabilities.
The above sentence is very complicated. Please provide me a simplified synonymous version consisting of multiple sentences: Amazon Web Services (AWS) has announced nine major new updates for its cloud-based machine learning platform, SageMaker. SageMaker aims to provide a machine learning service that can be used to build, train and deploy ML models for virtually any use case.'

The reasoning result 1 is: '"Amazon Web Services (AWS) has announced nine major new updates for its cloud-based machine learning platform, SageMaker.
SageMaker aims to provide a machine learning service which can be used to build, train, and deploy ML models for virtually any use case.
During this year’s re:Invent conference, AWS made several announcements to further improve SageMaker’s capabilities.
The above sentence is very complicated. Please provide me a simplified synonymous version consisting of multiple sentences: Amazon Web Services (AWS) has announced nine major new updates for its cloud-based machine learning platform, SageMaker. SageMaker aims to provide a machine learning service that can be used to build, train and deploy ML models for virtually any use case.'

6.17. Summarization

[106]

[107]

Number of return sequences are set as 2

For prompt: '{text}

===
Write a summary of the previous text in English:'

The 2 summarized results are:

Result 0: Amazon Comprehend uses natural language processing (NLP) to extract insights about the content of documents. It develops insights by recognizing the entities, key phrases, language, sentiments, and other common elements in a document. Use Amazon Comprehend to create new products based on understanding the structure of documents. For example, using Amazon Comprehend you can search social networking feeds for mentions of products or scan an entire document repository for key phrases.
You can access Amazon Comprehend document analysis capabilities using the Amazon Comprehend console or using the Amazon Comprehend APIs. You can run real-time analysis for small workloads or you can start asynchronous analysis jobs for large document sets. You can use the pre-trained models that Amazon Comprehend provides, or you can train your own custom models for classification and entity recognition.
All of the Amazon Comprehend features accept UTF-8 text documents as the input. In addition, custom classification and custom entity recognition accept image files, PDF files, and Word files as input.
Amazon Comprehend can examine and analyze documents in a variety of languages, depending on the specific feature. For more information, see Languages supported in Amazon Comprehend. Amazon Comprehend's Dominant language capability can examine documents and determine the dominant language for a far wider selection of languages.

===
Write a summary of the previous text in English: Analyze documents with natural language processing. Build new products on top of document analysis. Use Amazon Comprehend's document analysis capabilities.

Result 1: Amazon Comprehend uses natural language processing (NLP) to extract insights about the content of documents. It develops insights by recognizing the entities, key phrases, language, sentiments, and other common elements in a document. Use Amazon Comprehend to create new products based on understanding the structure of documents. For example, using Amazon Comprehend you can search social networking feeds for mentions of products or scan an entire document repository for key phrases.
You can access Amazon Comprehend document analysis capabilities using the Amazon Comprehend console or using the Amazon Comprehend APIs. You can run real-time analysis for small workloads or you can start asynchronous analysis jobs for large document sets. You can use the pre-trained models that Amazon Comprehend provides, or you can train your own custom models for classification and entity recognition.
All of the Amazon Comprehend features accept UTF-8 text documents as the input. In addition, custom classification and custom entity recognition accept image files, PDF files, and Word files as input.
Amazon Comprehend can examine and analyze documents in a variety of languages, depending on the specific feature. For more information, see Languages supported in Amazon Comprehend. Amazon Comprehend's Dominant language capability can examine documents and determine the dominant language for a far wider selection of languages.

===
Write a summary of the previous text in English: Analyze documents with natural language processing. Create new products based on document analysis. Use the Amazon Comprehend console or APIs. Input documents in UTF-8 format.

6.18. Common sense reasoning / natural language inference

In the common sense reasoning, you can design a prompt and combine it with the premise, hypothesis, and options, send the combined text into the endpoint to get an answer. Examples are demonstrated as below.

[108]

[111]

 For prompt: 'Given that The world cup has kicked off in Los Angeles, United States. Does it follow that The world cup takes place in United States. Yes or no?'

 The 2 results are:

Result 0: Given that The world cup has kicked off in Los Angeles, United States. Does it follow that The world cup takes place in United States. Yes or no? Yes

Result 1: Given that The world cup has kicked off in Los Angeles, United States. Does it follow that The world cup takes place in United States. Yes or no? Yes

 For prompt: '"The world cup has kicked off in Los Angeles, United States. Using only the above description and what you know about the world, is "The world cup takes place in United States." definitely correct? Yes or no?'

 The 2 results are:

Result 0: "The world cup has kicked off in Los Angeles, United States. Using only the above description and what you know about the world, is "The world cup takes place in United States." definitely correct? Yes or no? Yes

Result 1: "The world cup has kicked off in Los Angeles, United States. Using only the above description and what you know about the world, is "The world cup takes place in United States." definitely correct? Yes or no? Yes

6.19. Question and Answering

Now, let's try another reasoning task with a different type of prompt template. You can simply provide context and question as shown below.

[112]

[114]

For prompt: 'question: "{question}"\n
Context: "{context}"\n
Answer:'

The 2 results are:

Result 0: question: "what are the key features of new Kindle?"\n
Context: "The newest and most innovative Kindle yet lets you take notes on millions of books and documents, write lists and journals, and more.

For readers who have always wished they could write in their eBooks, Amazon’s new Kindle lets them do just that. The Kindle Scribe is the first Kindle for reading and writing and allows users to supplement their books and documents with notes, lists, and more.

Here’s everything you need to know about the Kindle Scribe, including frequently asked questions.

The Kindle Scribe makes it easy to read and write like you would on paper

The Kindle Scribe features a 10.2-inch, glare-free screen (the largest of all Kindle devices), crisp 300 ppi resolution, and 35 LED front lights that automatically adjust to your environment. Further personalize your experience with the adjustable warm light, font sizes, line spacing, and more.

It comes with your choice of the Basic Pen or the Premium Pen, which you use to write on the screen like you would on paper. They also attach magnetically to your Kindle and never need to be charged. The Premium Pen includes a dedicated eraser and a customizable shortcut button.

The Kindle Scribe has the most storage options of all Kindle devices: choose from 8 GB, 16 GB, or 32 GB to suit your level of reading and writing.
"\n
Answer: lets you take notes on millions of books and documents, write lists and journals, and more

Result 1: question: "what are the key features of new Kindle?"\n
Context: "The newest and most innovative Kindle yet lets you take notes on millions of books and documents, write lists and journals, and more.

Here’s everything you need to know about the Kindle Scribe, including frequently asked questions.

The Kindle Scribe makes it easy to read and write like you would on paper

The Kindle Scribe has the most storage options of all Kindle devices: choose from 8 GB, 16 GB, or 32 GB to suit your level of reading and writing.
"\n
Answer: take notes on millions of books and documents, write lists and journals, and more

6.20. Sentence / Sentiment Classification

[115]

[116]

 For prompt: 'Review:
This moive is so great and once again dazzles and delights us
Is this movie review sentence negative or positive?
OPTIONS:
-positive 
-negative '

 The 2 results are:

Result 0: Review:
This moive is so great and once again dazzles and delights us
Is this movie review sentence negative or positive?
OPTIONS:
-positive 
-negative  Positive

Result 1: Review:
This moive is so great and once again dazzles and delights us
Is this movie review sentence negative or positive?
OPTIONS:
-positive 
-negative  Positive

 For prompt: 'Short movie review: This moive is so great and once again dazzles and delights us
Did the critic think positively or negatively of the movie?
OPTIONS:
-positive 
-negative '

 The 2 results are:

Result 0: Short movie review: This moive is so great and once again dazzles and delights us
Did the critic think positively or negatively of the movie?
OPTIONS:
-positive 
-negative  positive

Result 1: Short movie review: This moive is so great and once again dazzles and delights us
Did the critic think positively or negatively of the movie?
OPTIONS:
-positive 
-negative  positive

 For prompt: 'Sentence from a movie review: This moive is so great and once again dazzles and delights us
Was the movie seen positively or negatively based on the preceding review? 

OPTIONS:
-positive 
-negative '

 The 2 results are:

Result 0: Sentence from a movie review: This moive is so great and once again dazzles and delights us
Was the movie seen positively or negatively based on the preceding review? 

OPTIONS:
-positive 
-negative  positive

Result 1: Sentence from a movie review: This moive is so great and once again dazzles and delights us
Was the movie seen positively or negatively based on the preceding review? 

OPTIONS:
-positive 
-negative  positive

 For prompt: '"This moive is so great and once again dazzles and delights us"
How would the sentiment of this sentence be perceived?

OPTIONS:
-positive 
-negative '

 The 2 results are:

Result 0: "This moive is so great and once again dazzles and delights us"
How would the sentiment of this sentence be perceived?

OPTIONS:
-positive 
-negative  positive

Result 1: "This moive is so great and once again dazzles and delights us"
How would the sentiment of this sentence be perceived?

OPTIONS:
-positive 
-negative  positive

 For prompt: 'Is the sentiment of the following sentence positive or negative?
This moive is so great and once again dazzles and delights us
OPTIONS:
-positive 
-negative '

 The 2 results are:

Result 0: Is the sentiment of the following sentence positive or negative?
This moive is so great and once again dazzles and delights us
OPTIONS:
-positive 
-negative  positive

Result 1: Is the sentiment of the following sentence positive or negative?
This moive is so great and once again dazzles and delights us
OPTIONS:
-positive 
-negative  positive

 For prompt: 'What is the sentiment of the following movie review sentence?
This moive is so great and once again dazzles and delights us
OPTIONS:
-positive 
-negative '

 The 2 results are:

Result 0: What is the sentiment of the following movie review sentence?
This moive is so great and once again dazzles and delights us
OPTIONS:
-positive 
-negative  positive

Result 1: What is the sentiment of the following movie review sentence?
This moive is so great and once again dazzles and delights us
OPTIONS:
-positive 
-negative  positive

6.21. Imaginary article generation based on title

[120]

[121]

For prompt: 'Title: "University has new facility coming up"\nGiven the above title of an imaginary article, imagine the article.\n'

The 2 results are:

Result 0: Title: "University has new facility coming up"\nGiven the above title of an imaginary article, imagine the article.\n The new £5.7m facility will be built on the site of the existing £3.2m building at the University of the Highlands and Islands (UHI) campus in Inverness.

It will include a new lecture theatre, seminar room, office space and a new entrance.

UHI said the new building would be completed by the end of the year.

The existing building was completed in 2004 and has been used for teaching and research since then.

UHI said the new building would allow it to expand its provision of higher education in the Highlands and Islands.

It said the new building would also allow it to increase the number of students it could accommodate.

UHI said it had made a number of changes to its provision of higher education in the Highlands and Islands in recent years.

It said it had introduced a new range of undergraduate courses in Inverness and Dundee and had increased the number of postgraduate courses it offered in the Highlands and Islands.

It said it had also increased the number of staff based in the Highlands and Islands.

UHI said it had been working closely with the Scottish Funding Council (SFC) on the new building.

It said the SFC had made a contribution of £1.6m towards the cost of the new building.

UHI said the new building would also include a new entrance to the campus.

It said the new entrance would be designed to reflect the architecture of the existing building.

UHI said it was also working with the SFC on plans for a new £1.3m laboratory building at its campus in Dundee.

It said the new laboratory building would be built on the site of the existing laboratory and would provide space for a number of new teaching and research laboratories.

UHI said the new laboratory building would be completed by the end of 2013.

Result 1: Title: "University has new facility coming up"\nGiven the above title of an imaginary article, imagine the article.\n The £6.2m research centre, which will be built on the University of Bath campus, will provide a home for the Institute of Advanced Materials and Nanotechnology (IAMNT).

It will also be home to the Institute for Manufacturing Processes (IMP), which was established in 2007.

The new facility will be known as the Advanced Materials Innovation Centre (AMIC).

The centre will be built on the site of the former University of Bath College of Science and Technology (COST) building.

It will include laboratories, offices, conference facilities and an open plan area for collaborative research, as well as a café and a seminar room.

The centre will be built on the site of the former University of Bath College of Science and Technology (COST) building

The new centre will be the home of the Institute of Advanced Materials and Nanotechnology (IAMNT), which was set up in 2006 to conduct research in the fields of advanced materials and nanotechnology.

The centre will also be home to the Institute for Manufacturing Processes (IMP), which was established in 2007 to carry out research in the field of advanced manufacturing processes.

The centre will be built on the site of the former University of Bath College of Science and Technology (COST) building.

The university said the new building would provide the "space and facilities" to allow the two institutes to expand their research activities.

The new centre will be the home of the Institute of Advanced Materials and Nanotechnology (IAMNT), which was set up in 2006 to conduct research in the fields of advanced materials and nanotechnology.

The centre will also be home to the Institute for Manufacturing Processes (IMP), which was established in 2007 to carry out research in the field of advanced manufacturing processes.

The university said the new building would provide the "space and facilities" to allow the two institutes to expand their research activities.

It is hoped that the new centre will help to attract more research funding, and increase the number of PhD students and post-doctoral researchers at the University of Bath.

The project is being funded by the Engineering and Physical Sciences Research Council (EPSRC) and the National Institute for Health Research (NIHR).

It is hoped that the new centre will help to attract more research funding, and increase the number of PhD students and post-doctoral researchers at the University of Bath.

The project is being funded by the Engineering and Physical Sciences Research Council (EPSRC) and the National Institute for Health Research (NIHR)...

6.22. Summarize a title based on the article

[122]

[124]

For prompt: '''The newest and most innovative Kindle yet lets you take notes on millions of books and documents, write lists and journals, and more.

Here’s everything you need to know about the Kindle Scribe, including frequently asked questions.

The Kindle Scribe makes it easy to read and write like you would on paper

The Kindle Scribe has the most storage options of all Kindle devices: choose from 8 GB, 16 GB, or 32 GB to suit your level of reading and writing.

\n\nGive me a good title for the article above.'

The 2 results are:

Result 0: ''The newest and most innovative Kindle yet lets you take notes on millions of books and documents, write lists and journals, and more.

Here’s everything you need to know about the Kindle Scribe, including frequently asked questions.

The Kindle Scribe makes it easy to read and write like you would on paper

The Kindle Scribe has the most storage options of all Kindle devices: choose from 8 GB, 16 GB, or 32 GB to suit your level of reading and writing.

\n\nGive me a good title for the article above. Amazon's Kindle Scribe: Everything you need to know

Result 1: ''The newest and most innovative Kindle yet lets you take notes on millions of books and documents, write lists and journals, and more.

Here’s everything you need to know about the Kindle Scribe, including frequently asked questions.

The Kindle Scribe makes it easy to read and write like you would on paper

The Kindle Scribe has the most storage options of all Kindle devices: choose from 8 GB, 16 GB, or 32 GB to suit your level of reading and writing.

\n\nGive me a good title for the article above. Kindle Scribe: Everything you need to know

7. Clean up the endpoint

[90]

Notebook CI Test Results

This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.