The Need for Encryption in LLMs

Encrypting LLMs is the answer if you want to use GenAI to:

1. Contextualize LLMs for enterprises

For internal GenAI use, enterprises would want to safeguard internal sensitive information while enabling GenAI to boost employee productivity. Customer-facing GenAI applications require custom data and robust privacy features on LLMs to safeguard proprietary data.

2. Navigate regulatory frameworks

Sharing private datasets with third parties (GenAI chatbots) can open a company to severe fines for non-compliance with regulations such as GDPR and CCPA. Countries such as Italy implemented a temporary ban on ChatGPT over data protection and privacy concerns. Several other European countries are assessing the tool’s compliance with the GDPR.

3. Fortify user interest

Cambridge Analytica comes to mind as one of the most controversial breaches of user privacy, where the political consulting company used the personal information of millions of Facebook (Meta) users for political advertising. Encrypting LLM data would be an ethical step in the general direction of ensuring data privacy.

4. Safeguard proprietary data

In an alarming incident, employees at Samsung Semiconductor used ChatGPT’s AI writer to resolve issues in their source code, feeding proprietary source code into a publicly available OpenAI, which would use the data to train itself. In two other instances at Samsung, proprietary information was provided to ChatGPT, raising IP and data privacy concerns and leading the company to ban the tool.

5. Unlock use cases

Most of all, it’s exciting to see encryption technologies coming to LLMs as they unlock potential use cases of GenAI. Developers get extremely restricted when privacy is insufficient or nearly impossible. Several use cases in industries such as finance, banking, law, and healthcare can’t come to fruition if data privacy remains a bottleneck.

Privacy concerns continue to abate innovation and commercialization of LLMs in enterprises for serious use cases particularly in sensitive industries, including finance and healthcare.

It’s not news that OpenAI’s ChatGPT refines its abilities using user data, and Google’s Bard has retention policies that contradict users’ data privacy expectations. This underscores a growing industry need for a user-centric approach to data privacy in GenAI implementation.

Various Encryption Solutions for LLMs

First, let’s consider encryption solutions that don’t work for modern technologies. Traditional encryption methods, such as AES and DES encryption, fall short in scenarios where data needs to be processed and safeguarded.

End-to-end encryption methods like those adopted by Web2 messaging services such as WhatsApp only protect data in transit. “It’s just a protection, a layer of encryption on top of the transmission”, says Pascal Paillier, CTO at Zama.

Why do these solutions fall short? If you want to process medical health data for predictive analysis, you’d want to preserve data privacy even during processing, which traditional encryption methods can’t achieve.

This is where Fully Homomorphic Encryption (FHE) comes in as a game-changer, allowing operations on data without ever exposing it.

Public LLMs are attack vectors unless you use FHE

FHE enables third parties to perform operations on underlying data without decrypting it. FHE is the third pillar of encryption. “We’ve had encryption in transit. We had encryption at rest. We never had encryption at use.”, says Paillier. FHE represents encryption at use.

This is what we are solving for at Fhenix, along with our partner, Zama. Our mission is to empower developers and data scientists to leverage FHE without having to learn cryptography to secure LLMs.

Here’s how FHE will end-to-end encrypt LLMs and ChatGPT:

• Encrypt the query and contextual data using a secret key known only to the user
• Send the encrypted prompt to the service provider running the LLM
• Compute the LLM on encrypted data and produce an encrypted response
• Send the encrypted response to the user, who then decrypts it using their key

This way, FHE would encrypt all communications between the user and the GenAI bot so that the application owners don’t retain and misuse users’ sensitive data. FHE would enable better use cases of GenAI by eliminating the privacy issue and permitting users to feed custom datasets to derive the most value from GenAI.

While FHE is the solution, the journey to implementing it isn’t without its hurdles. Let’s explore the challenges and how we’re tackling them.

Challenges and the Road Ahead

While FHE computation performance has improved by 20x in the last 3 years, we still have a ways to go before FHE encrypts LLMs. Estimates show that generating one encrypted LLM token would need up to 1 billion FHE operations, due to which FHE remains cost-prohibitive.

However, three significant trends are contributing to FHE readiness for LLMs:

• Compression techniques are making LLMs faster, resulting in less data to compute homomorphically. This will likely bring a 2x performance improvement for FHE.
• The cryptography behind FHE is improving, and we can realistically expect a 5x acceleration in the next five years.
• Several companies, including Duality, Intel, and IBM, are currently developing custom ASICs and FPGAs optimized for bootstrapping and homomorphic operations. These companies are targeting a 1,000x speedup for their first generation (planned for 2025) and up to 10,000x for their second generation. Consequently, you’d only need about 5 FHE accelerators to run an encrypted LLM, on par with the number of GPUs you need today for non-encrypted LLMs.

Our focus at Fhenix so far has been to make FHE possible. Next, we aim to make it feasible.

Challenge 1: Nascent FHE Schemes, Libraries and Compilers

First, let’s look at each term in turn:

1. Schemes: standardized and systematic approaches that help developers manage data or code to efficiently solve problems. This could include architectural designs, patterns, and best practices.
2. Libraries: collections of pre-written code that programmers can use to efficiently perform common tasks or access specific functionalities.
3. Compilers: these translate high-level programming languages into low-level machine code that can be easily executed on a blockchain platform.  Developers require a fully functional FHE compiler that handles complexities on their behalf, such as parameter selection.

Various challenges exist. For instance, existing FHE schemes (particularly TFHE) are around 1000x slower compared to plain computation. Developers need easy-to-use front-end libraries that abstract away the low-level FHE complexities, and a fully functional FHE compiler that handles complexities on their behalf.

We’ll cover the challenges associated with each, before showcasing how the appropriate solution is nearly here.

FHE Schemes

To begin implementing FHE into one’s application, the appropriate scheme must be chosen, which can be summarized in the table below.

While not all terms may be familiar to the reader, they may be worth further exploration if of interest.

Each of the above has inherent trade-offs, such as BGV/BFV being ideal for DeFi, but requiring knowledge of a circuit’s depth in advance. While FHEW/TFHE could also be used in DeFi, BGV/BFV are considered more efficient due to batching operations whereby multiple computations are batched together.

Developers must choose their scheme very carefully as it will impact other design decisions and future application performance, though for blockchain purposes, Exact Arithmetic tends to be most beneficial.

FHE Libraries

Depending on the chosen scheme, there are different libraries available to aid in the front-end programming experience.

Each scheme has a different relationship with each library, and the developer must also set parameters for their choices.

Another challenge for developers is that writing FHE programs requires a mental shift. For instance, one cannot write a branch (if-else) depending on a variable in the program, because programs cannot directly compare the variables like plain data. They also cannot write loops in their code for the same reason.

Solution: Web3 FHE Libraries & Compilers

Both FHE libraries and compilers have already been developed by leading technology firms such as Google and Microsoft, but primarily use the CKKS scheme which is focused on Machine Learning. These cannot adequately support ZKP schemes nor the intricacies of program chaining, which involves the sequential connection of multiple computer programs or software modules. Instead, DeFi-specific schemes such as BFV/BGV are needed. Fortunately, these Web3 FHE libraries and compilers now exist and are addressing all of the above challenges.

1. FHE Libraries

Zama’s open source TFHE-rs (Fully Homomorphic Encryption over the Torus) library enables the creation of FHE-based smart contracts in Solidity for Zama’s fhEVM. Additionally, there are ongoing developments in the TFHE ecosystem, with libraries like TFHE Rust and TFHE C++ (though the latter is still in the process of being fully developed).

2. FHE Compilers

Sunscreen FHE Compiler is a recently emerged Web3 compiler designed for DeFi use cases, allowing developers to program in Rust. It relies on Microsoft’s SEAL library and utilizes the BFV scheme.  Sunscreen simplifies the complexities mentioned earlier by offering features such as automatic parameter selection, circuit setup, data encoding, key selection, and more. Notably, it has demonstrated high performance among FHE compilers.

Moreover, the ongoing development of other FHE compilers signals positive advancements in this field, with the expectation of continued optimizations in the future.

Challenge 2: Secure Threshold Decryption

Given that FHE provides the computation of encrypted data, there’s a need to have a secure encryption and decryption key for the data to start its journey or arrive at its destination.

Therefore, there’s a need for an extremely secure and yet efficient threshold decryption protocol. This refers to a protocol or mechanism used to collectively decrypt data in a secure and distributed manner. It ensures that multiple parties must cooperate to decrypt information, making it difficult for any single entity to corrupt the process.

Currently, there are two bottlenecks here:
1. Overhead: FHE-based computation is complex and therefore requires high-performance nodes, which reduces the potential number of eligible nodes.
2. Latency: While more nodes improve decentralization, this also increases latency.

We therefore require a decryption technique that a) minimizes the chances of collusion (given that we may have a smaller set of nodes), b) is permissionless for new nodes, and c) is low-latency.

Solution: Threshold Decryption or Dynamic MPC

We see a few potential threshold decryption solutions:
1. N/2 threshold decryption: this means that more than half of the participants must cooperate in the decryption process, making it harder for malicious actors to collude.
2. Hardware Security Module (HSM) threshold decryption: these are specialized devices designed to protect cryptographic keys and perform secure operations, which adds an extra layer of security.
3. Trusted Execution Environment (TEE) threshold decryption: a secure environment for executing code involving sensitive data.

Alternatively, we could use Multiparty Computation (MPC), the common wallet security technique. An example of a possible technique that could be used for FHE is a variation of a traditional MPC implementation known as the 2PC-MPC algorithm. It requires all validators to participate, with 2/3 of validators (plus the user) required to generate a signature. One potential trade-off is that MPC may be more computationally complex and with more participants, which affects latency. This is an area of on-going research and may improve over time.

In summary, both threshold decryption and MPC variations such as the 2PC-MPC algorithm are being rapidly developed and show great promise for on-chain FHE.

Penn Blockchain Week: February 23-24

Penn Blockchain Week is one of the leading academic blockchain events. It includes speakers from various top-tier DeFi projects such as Eclipse, Berachain, and Gauntlet, as well as VC firms such as Bain Capital, Portal Ventures, and Union Square Ventures.

Learn more about Penn Blockchain Week.

ETH Denver: February 23 – March 6

Fhenix will be either hosting or attending four different ETHDenver events:

• DeFi Day: Organized by Dystopia Labs, this February 26th event will showcase talks from various DeFi projects, including Fhenix. More information shortly.

• Private Research Dinner: On February 27th we will be hosting a private research dinner for encryption leaders. More information shortly.

• Encryption Day: On February 28th we are bringing together the top minds in the encryption space, including the Founders of Fhenix, Zama, Arbitrum, EigenLayer, and Bankless, as well as researchers from Flashbots, Starkware, and more. Click here to register.

• ETHDenver Hackathon: Renowned for incubating top Ethereum projects, this hackathon stands out as a highlight of the year. Learn more here.

ETHGlobal London: March 15-17

ETHGlobal events are worldwide and focused on fostering a global Ethereum community, with speeches, hackathons, workshops, and more.

Learn more about ETHGlobal.

ETHSeoul: March 29-31

This annual event in Seoul, Korea is known to attract leading Ethereum researchers, developers, and community members from the APAC region.

Learn more about ETHSeoul.