In the AI era, there is an ever-growing demand for information that is both comprehensive and easily digestible. That’s why Conveyer created the AI Conversion Engine (ACE), a transformative solution that redefines our approach to data processing — and opens the door to powerful new ways of leveraging complementary technologies such as generative AI (GenAI).
What truly sets ACE apart is its ability to convert unstructured or semi-structured data into a meticulously structured format called “Topics” — specific, precisely defined categories or themes that cohesively organize the data into a structured whole.
This new approach represents a fundamental shift in how we engage with data. By precisely categorizing and united previously unstructured or loosely structured information with structured information, this technology makes information more readily accessible, more interactive, and more engaging. In addition to generating topics, ACE can support the creation of all GenAI artifacts — including but not limited to FAQs, sentiment analysis, classification, keywords, entities, and many more — further enhancing the depth and breadth of data exploration and comprehension.
How Does ACE Unlock Value for Businesses?
Consider a practical example, where a company uses ACE to transform unstructured or semi-structured data into structured Topics. With the aid of GenAI, it then goes a step further by generating sentiment analysis for the topic data. This process not only enhances data organization, but also extracts valuable insights from the structured information, shedding light on the sentiments embedded within.
Let’s say the organization has extensive volumes of customer data (emails, call recordings, support logs, etc.), and they want to use that unstructured data to identify sentiment patterns to ultimately enhance overall customer satisfaction. The problem is, this data is typically unstructured, sprawling, and seemingly all but impossible to leverage effectively. For most businesses, trying to derive insights from such data is not worth the investment due to GPU shortage and high processing costs.
That’s where Conveyer AI steps in. Conveyer’s ACE can transform this complex data into a structured, coherent format and deliver tailored insights meticulously aligned with the organization’s specific operational needs and objectives.
After transforming unstructured data into a structured topic format and generating artifacts using GenAI tailored to each topic, we can now delve into illustrating the sentiment analysis for the data.
The sentiment analysis provides insights into whether the content carries a positive, negative, or neutral tone. Each topic receives a sentiment analysis score, which is a numeric representation of the strength of that sentiment within the topic. A negative score corresponds to a negative sentiment, indicating a more pessimistic tone, while a positive score signifies a positive sentiment, reflecting a more optimistic tone. A score of zero is indicative of a neutral sentiment, suggesting an absence of strong positive or negative emotions. This detailed analysis aids in understanding the nuanced emotional context associated with each specific theme or subject.
Once these artifacts, including sentiment scores, are obtained, the next step in addressing business challenges involves visualizing the data. In our approach, we utilize vector data visualization, facilitated by the data structuring platform Atlas Nomic AI.
This process begins with textual embedding generation — a crucial step that transforms textual information into numerical vectors. These vectors are then utilized to visualize the vector representation of the text data. This comprehensive approach, which integrates both text embedding and sentiment scores, adds a layer of depth to our understanding, enabling us to extract valuable insights from the structured and sentiment-analyzed data. The vectorization process enhances our ability to unravel complex patterns and relationships within the data, fostering a more holistic understanding of the information at hand.
The following code demonstrates the process of visualizing the Pinecone vector database index in Atlas.
"""
Visualizing your pinecone vector database index in Atlas
"""
import pinecone
import numpy as np
from nomic import atlas
import nomic
pinecone.init(api_key='YOUR PINECONE API KEY', environment='us-east1-gcp')
nomic.login('YOUR NOMIC API KEY')
#create and insert embeddings into your pinecone index
pinecone.create_index("quickstart", dimension=128, metric="euclidean", pod_type="p1")
index = pinecone.Index("quickstart")
num_embeddings = 999
embeddings_for_pinecone = np.random.rand(num_embeddings, 128)
index.upsert([(str(i), embeddings_for_pinecone[i].tolist()) for i in range(num_embeddings)])
#now pull the embeddings out of pinecone by id
vectors = index.fetch(ids=[str(i) for i in range(num_embeddings)])
ids = []
embeddings = []
for id, vector in vectors['vectors'].items():
ids.append(id)
embeddings.append(vector['values'])
embeddings = np.array(embeddings)
atlas.map_embeddings(embeddings=embeddings, data=[{'id': id} for id in ids], id_field='id')
You can access the Sentiment Analysis | Atlas Nomic AI here.
The showcased data and its visualizations underscore the potency of Conveyer’s state-of-the-art technology. By transforming vast, unstructured datasets into structured formats, Conveyer empowers businesses to derive actionable insights from complex information sources such as customer emails and call recordings. This capability is particularly valuable when aiming to uncover customer satisfaction insights from disparate, unstructured everyday data sources.
Conveyer offers a powerful solution for tackling intricate business challenges and harnessing the potential within data-driven environments. Get in touch with the Conveyer team today to learn more.
