Enhancing Question and Answer with openAI, A Step-by-Step Guide

Searching for information online can sometimes feel like looking for a needle in a haystack, but advanced AI models, like GPT, can help augment and refine this process efficiently. In this blog, we’ll walk you through a hybrid approach to augment search systems using GPT, providing an example using the News API.

Overview

Our approach combines mimicking human browsing and retrieval with embeddings to optimize the search process. It involves three steps:

1. Search: Execute multiple search queries generated from a user’s question.
2. Re-rank: Calculate semantic similarity between search results and a hypothetical answer, then re-rank the results.
3. Answer: Generate a concise answer using the top re-ranked search results.

Step-by-Step Implementation

Setup

First, acquire API keys for OpenAI and the News API. Install necessary Python dependencies like openai, requests, numpy, tqdm, and json.

%env NEWS_API_KEY = YOUR_NEWS_API_KEY
import openai, os, requests, json, tqdm
from datetime import date, timedelta
from numpy import dot
from IPython import display

Step 1: Search

Start by obtaining the user’s question, then use GPT to generate potential search queries related to the question.

USER_QUESTION = "Who won the NBA championship? And who was the MVP?"

def generate_queries(user_question):
    QUERIES_INPUT = f"""
    You have access to a search API that returns recent news articles. Generate an array of search queries that are relevant to this question. Be creative.
    User question: {USER_QUESTION}
    Format: {{"queries": ["query_1", "query_2", "query_3"]}}
    """
    queries = json_gpt(QUERIES_INPUT)["queries"]
    queries.append(USER_QUESTION)
    return queries

queries = generate_queries(USER_QUESTION)

where json_gpt is talking to openai with system role to make sure the results follow json format.

# Helper functions
def json_gpt(input: str):
    completion = openai.ChatCompletion.create(
        model=GPT_MODEL,
        messages=[
            {"role": "system", "content": "Output only valid JSON"},
            {"role": "user", "content": input},
        ],
        temperature=0.5,
    )

    text = completion.choices[0].message.content
    parsed = json.loads(text)

    return parsed


def embeddings(input: list[str]) -> list[list[str]]:
    response = openai.Embedding.create(model="text-embedding-ada-002", input=input)
    return [data.embedding for data in response.data]

Then fetch search results from the News API for each query:

def search_news(queries):
    articles = []
    for query in tqdm.tqdm(queries):
        result = search_news_api(query)
        if result["status"] == "ok":
            articles += result["articles"]
    return list({article["url"]: article for article in articles}.values())

articles = search_news(queries)

Step 2: Re-rank

Generate a hypothetical answer to the user’s question to use as a basis for re-ranking:

def generate_hypothetical_answer(user_question):
    HA_INPUT = f"""
    Generate a hypothetical answer to the user's question. Pretend you have all the information needed.
    User question: {USER_QUESTION}
    Format: {{"hypotheticalAnswer": "hypothetical answer text"}}
    """
    return json_gpt(HA_INPUT)["hypotheticalAnswer"]

hypothetical_answer = generate_hypothetical_answer(USER_QUESTION)

Compute the semantic similarity between each search result and the hypothetical answer, then re-rank the results:

def rank_articles(articles, hypothetical_answer):
    article_embeddings = embeddings([f"{article['title']} {article['description']} {article['content'][0:100]}" for article in articles])
    hypothetical_answer_embedding = embeddings([hypothetical_answer])[0]
    cosine_similarities = [dot(hypothetical_answer_embedding, article_embedding) for article_embedding in article_embeddings]
    scored_articles = zip(articles, cosine_similarities)
    return sorted(scored_articles, key=lambda x: x[1], reverse=True)

scored_articles = rank_articles(articles, hypothetical_answer)

Step 3: Answer

With the top-ranked articles, construct a comprehensive answer to the user’s question:

def generate_final_answer(user_question, top_articles):
    formatted_top_results = [{"title": article["title"], "description": article["description"], "url": article["url"]} for article, _score in top_articles]
    ANSWER_INPUT = f"""
    Generate an answer to the user's question based on the given search results.
    TOP_RESULTS: {formatted_top_results}
    USER_QUESTION: {USER_QUESTION}
    """
    completion = openai.ChatCompletion.create(model="gpt-3.5-turbo", messages=[{"role": "user", "content": ANSWER_INPUT}], temperature=0.5, stream=True)
    text = ""
    for chunk in completion:
        text += chunk.choices[0].delta.get("content", "")
    return text

answer = generate_final_answer(USER_QUESTION, scored_articles[0:5])

Wrapping Up

This approach efficiently sifts through a vast array of information, providing accurate and relevant answers to user questions with relatively low latency. It can be easily integrated into existing search systems, making it a valuable tool for enhancing the search experience.

code link

github link