Jane Edwards
FedTech Magazine reported Thursday that retrieval augmented generation, or RAG, could help improve the value that generative artificial intelligence brings to agencies by enabling large language models to access an agency’s databases and other external information sources to produce contextual responses to user prompts.
Peter Guerra, group vice president for data and artificial intelligence at Oracle, said that RAG leverages a process called embedding to do that.
“Embedding requires using an LLM that is specifically designed for the process,” said Guerra. “It takes the information stored in the source text, turns that into a vector representation and then stores that in a vector database. When the RAG actually does the query, it’s using the native language that the LLM understands, and then it can use that against the vector database.”
Setting Up a RAG Workflow Through Embedding Process
The Oracle executive noted that embedding is a key step in setting up a RAG workflow.
“You have to create the vector embeddings for your data and then store that in the database,” Guerra says. “Then it’s simply pointing the RAG application at that data and enabling that user query through good sanitation and other application security practices to be able to do the RAG effectively.”
Protecting Sensitive Data
According to Guerra, RAG could help agencies “control some of the security flow of how that data is being used.”
“It allows you to keep the data where it is, not exposing it to risk by moving it. You keep it secure, but at the same time you can ask questions of the data and get the most accurate, best representation back,” he added.
Implementing Robust Safeguards
RAG requires extensive assessment before and after the approach goes into operation.
“It’s important to explain what users should and shouldn’t do,” Guerra said. “And also ensure robust safeguards are implemented to prevent people from submitting queries that will return problematic responses.”
