The technical workflow of the LLM Firewall involves performing the tokenization, detecting and filtering sensitive information, and interacting with LLMs.

Current Approach

Let’s take the current scenario on how currently users use the AI applications.

Example Interaction

Let’s consider a commonly used application where the user tries to improve wording with the help of ChatGPT.

I am writing a mail to my colleague. Rewrite.'

"Dear Dr. Smith,
This is a letter about our mutual patient named as John. John is residing in Springfield. John is diagnosed with Type 2 Diabetes and hypertension." write well

As you can see from the above figure, the content with sensitive information such as patient name, location, and diagnosed disease gets leaked on the public domain when the user tries to correct the wordings.

The current approach affects privacy in many ways. Let’s take a few examples:

1. Somewhere if another user asks to create something about john and disease, LLMs could write a sentence with John, and Type 2 Diabetes
2. Within the same organization, information related to John could unintentionally be leaked to a different user.
3. Databases storing embeddings could include John and the diseases. This puts them at risk in case of a data attack.

As we have seen in the previous page, LLM Firewall preprocesses the personal information in the user text and converts into generic information, and then sends it to LLM cloud/Internet.

Let’s consider the same example.

I am writing a mail to my colleague. Rewrite.
" Dear Dr. Smith,
This is a letter about our mutual patient named as John. John is residing in
Springfield. John is diagnosed with Type 2 Diabetes and hypertension." write well

LLM Firewall changes the above text to:

Preprocessed text:

I am writing a mail to my colleague. Rewrite.
Dear ZK_NAME_MEDICAL_PROFESSIONAL,
This is a letter about our mutual patient named as ZK_NAME_GIVEN.
ZK_NAME_GIVEN is residing in ZK_LOCATION_CITY.
ZK_NAME_GIVEN is diagnosed with ZK_CONDITION and ZK_CONDITION."

GPT Response:

Dear ZK_NAME_MEDICAL_PROFESSIONAL,
This letter regards our mutual patient, ZK_NAME_GIVEN. ZK_NAME_GIVEN is
currently residing in ZK_LOCATION_CITY.
They have been diagnosed with the following conditions:
ZK_CONDITION
ZK_CONDITION
If you have any questions or need further information, please feel free to reach out.
Sincerely,
[Your Name]
[Your Title]
[Your Contact Information]

As you can see, the task is completed with the help of GPT without exposing any sensitive data. Your data is protected. Sensitive data is filtered.