Insights Generator Notebook
End_to_end_Solutionsazure-openai-samplesInsightsGenerator
Export
[1]
Product Reviews Insights Generator
Takes in product reviews and generates high level and statistical insights.
First, some setup.
[2]
[3]
Project metadata:
{'name': 'headphones-1', 'productCategory': 'headphones', 'productName': 'Contoso headphones'}
Reading json from file: sample_reviews/contoso_headphones.json
Identify topics being discussed in the reviews.
First get aspects (i.e. topics) for each batch of reviews. Batching is done to standardize the aspect names across reviews.
[4]
Extracting sentiment aspects for 16 reviews
Starting 4 parallel extractors
Extracted sentiment aspects for 16 reviews in 3.282043933868408 seconds
Done.
Sentiment Aspects extracted for reviews.
Sample review with sentiment aspects:
{
"review_text": "review: I recently purchased the Contoso 100 headphones and I'm really impressed with the sound quality. The noise cancellation is great and I can block out all outside noise. My only complaint is the battery life, which isn't as long as I'd like it to be.\nreview: I've been using the Contoso 100 headphones for a few weeks now and I'm really pleased with them. The sound quality is excellent and the noise cancellation works great. The only downside is the battery life, which seems to run out too quickly.\nreview: I recently bought the Contoso 100 headphones and I'm very happy with them. The sound quality is great and the noise cancellation is top notch. The only issue I have is with the battery life, which doesn't seem to last as long as I'd like.\nreview: I've been using the Contoso 100 headphones for a few weeks now and I'm really impressed. The sound quality is excellent and the noise cancellation works really well. My only complaint is with the battery life, which doesn't seem to last very long.\nreview: I recently bought the Contoso 100 headphones and I'm really happy with them. The sound quality is great and the noise cancellation works really well. My only issue is with the battery life, which seems to run out too quickly.\nreview: I've been using the Contoso 100 headphones for a few weeks now and I'm really pleased with them. The sound quality is excellent and the noise cancellation works great. My only complaint is with the battery life, which doesn't seem to last very long.",
"sentiment_aspects": [
[
"sound quality",
"positive"
],
[
"noise cancellation",
"positive"
],
[
"battery life",
"negative"
]
]
}
Writing json to file: reviews_with_sentiment_aspects.json
Next get the most frequent aspects, by counting occurrences across batches.
[5]
Top aspects and scores:
[
[
"sound quality",
3
],
[
"noise cancellation",
3
],
[
"battery life",
3
]
]
Summarize the Top Topics (Aspects)
An aspect (i.e. topic) is summarized both in NL (natural language) and statistically.
- Extracting sentiment + keyphrases for each review, for that aspect
- Statistical summary of sentiments is done
- Keyphrases are summarized as a NL (natural language) summary.
This aspect level summary is a lot more accurate and scalable than summarizing all aspects across all reviews.
[6]
Extracting keyphrases for sound quality Done. Extracting keyphrases for noise cancellation Done. Extracting keyphrases for battery life Done. Extracted all statistics
[7]
Sample review with top aspect sentiments:
{
"location": "US",
"profile_name": "James Smith",
"review_text": "I recently purchased the Contoso 100 headphones and I'm really impressed with the sound quality. The noise cancellation is great and I can block out all outside noise. My only complaint is the battery life, which isn't as long as I'd like it to be.",
"top_aspect_sentiments": {
"sound quality": "positive",
"noise cancellation": "positive",
"battery life": "negative"
}
}
[8]
Aspect level summary, example for sound quality:
{
"positive": 13,
"unmentioned": 2,
"negative": 1,
"overall_sentiment": "positive",
"percentage_positive": 92.85714285714286,
"keyphrases": [
{
"keyphrases": [
" Great sound quality",
" Excellent noise cancellation",
" Battery life not as long as desired"
],
"profile_name": "James Smith",
"location": "US"
},
{
"keyphrases": [
" Excellent sound quality",
" Great noise cancellation"
],
"profile_name": "Jessica Johnson",
"location": "US"
},
{
"keyphrases": [
" Great sound quality",
" Top notch noise cancellation"
],
"profile_name": "Adam Williams",
"location": "US"
},
{
"keyphrases": [
" Excellent sound quality",
" Effective noise cancellation"
],
"profile_name": "Samantha Jones",
"location": "US"
},
{
"keyphrases": [
" Great sound quality",
" Effective noise cancellation"
],
"profile_name": "Matthew Brown",
"location": "US"
},
{
"keyphrases": [
" Excellent sound quality",
" Great noise cancellation"
],
"profile_name": "Emily Davis",
"location": "US"
},
{
"keyphrases": [
" Great sound quality",
" Excellent noise cancellation",
" Battery life not as long as desired"
],
"profile_name": "John Taylor",
"location": "US"
},
{
"keyphrases": [
" Excellent sound quality",
" Great noise cancellation"
],
"profile_name": "Olivia White",
"location": "US"
},
{
"keyphrases": [
" Great sound quality",
" Top notch noise cancellation"
],
"profile_name": "Daniel Moore",
"location": "US"
},
{
"keyphrases": [
" Excellent sound quality",
" Effective noise cancellation"
],
"profile_name": "Charlotte Anderson",
"location": "US"
},
{
"keyphrases": [
" Great sound quality",
" Effective noise cancellation"
],
"profile_name": "William Thomas",
"location": "US"
},
{
"keyphrases": [
" Impressive sound quality",
" Effective noise cancellation capabilities"
],
"profile_name": "Joshua Jackson",
"location": "US"
},
{
"keyphrases": [
" Excellent sound quality",
" Effective noise cancellation"
],
"profile_name": "Amelia Harris",
"location": "US"
}
],
"aspect_summary": "The comments indicate that the sound quality is excellent with great noise cancellation.",
"aspect_action_items": null
}
Overall NL (Natural Language) Summaries
The aspect level summaries from previous step are clubbed into high level NL (natural language) summaries.
[9]
Extracted NL version of statistical summary.
{
"overall": "The comments indicate that the sound quality and noise cancellation of the headphones are good, but the battery life is poor.",
"highlights": "The comments about the headphones indicate that they have excellent sound quality and great noise cancellation, with the latter being particularly impressive, as it is able to block out all outside noise.",
"lowlights": "The comments about the headphones indicate that the battery life is not satisfactory, as it runs out quickly and does not last as long as desired.",
"action_items": {
"battery life": "Specific issues regarding battery life include that it is not as expected, runs out quickly, and does not last as long as desired. To improve battery life, the manufacturer could use higher quality components, optimize the power management system, and reduce the power consumption of the device"
}
}