Give a try
Why need an Enterprise AI solution
Artificial intelligence (AI) has become essential for businesses to streamline operations and improve overall efficiency. AI-powered tools can help companies automate time-consuming tasks, gain insights from vast data and make informed decisions. Due to the data security, privacy and customization, business needs an Enterprise AI solution that can work like ChatGPT but are free, open-source and running on consumer hardware.
What is Alpaca-Lora?
Alpaca-LoRA is a 7B-parameter LLaMA model finetuned to follow instructions. It is trained on the Stanford Alpaca dataset and makes use of the Huggingface LLaMA implementation. Alpaca-LoRA uses Low-Rank Adaptation(LoRA) to accelerate the training of large models while consuming less memory.
Why chooses Alpaca-Lora?
Alpaca-Lora is faster and uses less memory, which means it can run on consumer hardware. The output is much smaller (megabytes, not gigabytes). Business can combine multiple fine-tuned models together at runtime.
For more details, please check Git Alpaca-LoRA repository and The official demoBuild financial dataset
Preparing the high-quality dataset is the key step to build Enterprise large language model. This financial dataset contains 68.9K instruction-following data. It is a combination of Stanford's Alpaca and FiQA with another 1.3k pairs custom dataset generated using GPT3.5.
Prerequisites
The following lists the recommendation hardware and software for the business to run Alpaca Lora.
A list of tools need to be installed, such as: Anaconda for your own python version; setuptools, pip etc.
1: wget https://repo.anaconda.com/archive/Anaconda3-5.3.0-Linux-x86_64.sh sh Anaconda3-5.3.0-Linux-x86_64.sh 2: wget https://files.pythonhosted.org/packages/26/e5/9897eee1100b166a61f91b68528cb692e8887300d9cbdaa1a349f6304b79/setuptools-40.5.0.zip unzip setuptools-40.5.0.zip cd setuptools-40.5.0/ python setup.py install 3: wget https://files.pythonhosted.org/packages/45/ae/8a0ad77defb7cc903f09e551d88b443304a9bd6e6f124e75c0fbbf6de8f7/pip-18.1.tar.gz tar -xzf pip-18.1.tar.gz cd pip-18.1 python setup.py install 4: Create own version of python conda create -n alpaca python=3.9 conda activate alpaca
Financial LLM Python Implementation
We will create a Python environment to run Alpaca-Lora on our local machine.
1: Cloning GitHub Repository
$ git clone https://github.com/tloen/alpaca-lora.git $ cd .\alpaca-lora\
2: Install dependencies
$ pip install -r requirements.txt
3: Training
The python file named finetune.py contains the hyperparameters of the LLaMA model, like batch size, number of epochs, learning rate (LR), etc., which you can play with.
$ python finetune.py \
--base_model 'decapoda-research/llama-7b-hf' \
--data_path='./data/finance/data/alpaca-finance.json' \
--output_dir='./output/finance' \
--num_epochs 10
4: Running the Model
The python file named generate.py will read the Hugging Face model and LoRA weights from tloen/alpaca-lora-7b. It runs a user interface using Gradio, where the user can write a question in a textbox and receive the output in a separate textbox.
python generate.py \
--load_8bit \
--base_model 'decapoda-research/llama-7b-hf' \
--lora_weights 'output/finance'
Open https://localhost:7860 in the browser for quick test
Docker Setup & Inference
Build the container image:
docker build -t alpaca-lora .
Run the container (you can also use finetune.py and all of its parameters as shown above for training):
docker run --gpus=all --shm-size 64g -p 7860:7860 -v /.cache:/root/.cache --rm alpaca-lora generate.py \
--load_8bit \
--base_model 'decapoda-research/llama-7b-hf' \
--lora_weights 'tloen/alpaca-lora-7b'
Open https://localhost:7860 in the browser
