From 15c8ca96b291d07d429cec0504b0793853ffe5e4 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 18 Apr 2023 20:01:26 +0000 Subject: [PATCH 001/155] docs: clarify upstream source and license attribution in models README --- src/instructlab/train/lora_mlx/models/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/instructlab/train/lora_mlx/models/README.md b/src/instructlab/train/lora_mlx/models/README.md index 9c268812ef..9e9507c5a5 100644 --- a/src/instructlab/train/lora_mlx/models/README.md +++ b/src/instructlab/train/lora_mlx/models/README.md @@ -1,6 +1,6 @@ # Notice -The code in this folder is modified from [here](https://github.com/ml-explore/mlx-examples/tree/main/lora/models) with the following license. +The code in this folder is modified from the [mlx-examples lora/models](https://github.com/ml-explore/mlx-examples/tree/main/lora/models) directory, distributed under the MIT License reproduced below. ```license MIT License @@ -24,4 +24,4 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -``` \ No newline at end of file +``` From d6c906bc2cc4737c389beadcd6ac55d775e389f3 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 19 Apr 2023 17:01:32 +0000 Subject: [PATCH 002/155] Add heading to SECURITY.md --- SECURITY.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/SECURITY.md b/SECURITY.md index b37d8f1cbe..06838f5ec7 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1 +1,3 @@ +# Security Policy + You can find information on how to report a potential security vulnerability, as well as where to subscribe to receive security alerts, on the project's [Security Page](https://github.com/instructlab/.github/blob/main/SECURITY.md). From b6b7039ed8943946191316458d81e87bd6035027 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 20 Apr 2023 14:45:02 +0000 Subject: [PATCH 003/155] docs: clarify maintainers link in MAINTAINERS.md --- MAINTAINERS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 1b526af073..5d436787ed 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -1,3 +1,3 @@ # InstructLab Maintainers -For a complete list of InstructLab project Maintainers, see [Maintainers](https://github.com/instructlab/community/blob/main/MAINTAINERS.md). \ No newline at end of file +For a complete list of InstructLab project Maintainers, see the [Maintainers list](https://github.com/instructlab/community/blob/main/MAINTAINERS.md) in the `instructlab/community` repository. From 9ab04a93e6b3bf597ba66b62bb7d5230ace97e32 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 21 Apr 2023 18:22:21 +0000 Subject: [PATCH 004/155] docs: clarify SECURITY.md link references InstructLab project --- SECURITY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/SECURITY.md b/SECURITY.md index 06838f5ec7..5c45c64706 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,3 +1,3 @@ # Security Policy -You can find information on how to report a potential security vulnerability, as well as where to subscribe to receive security alerts, on the project's [Security Page](https://github.com/instructlab/.github/blob/main/SECURITY.md). +You can find information on how to report a potential security vulnerability, as well as where to subscribe to receive security alerts, on the InstructLab project's [Security Page](https://github.com/instructlab/.github/blob/main/SECURITY.md). From dbdb33761ca7678d66b2746df94ea66ecd7cf87d Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 28 Apr 2023 00:16:34 +0000 Subject: [PATCH 005/155] docs: recommend using a Python venv for dev setup --- CONTRIBUTING/CONTRIBUTING.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CONTRIBUTING/CONTRIBUTING.md b/CONTRIBUTING/CONTRIBUTING.md index 79d97156ff..dfa1e46006 100644 --- a/CONTRIBUTING/CONTRIBUTING.md +++ b/CONTRIBUTING/CONTRIBUTING.md @@ -88,6 +88,8 @@ If you want to test the `ilab` binary, you can install `ilab` and all dependenci pip install . ``` +It is recommended to work inside a Python [virtual environment](https://docs.python.org/3/library/venv.html) (e.g., `python -m venv venv && source venv/bin/activate`) to keep project dependencies isolated from your system Python. + ### Testing Before pushing changes to GitHub, you need to run the tests as shown below. They can be run individually as shown in each sub-section From 67211e59a16b9a0fbea924432620b682cca813b8 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 1 May 2023 18:04:16 +0000 Subject: [PATCH 006/155] Include pylint informational (I####) codes in warning matcher --- .github/workflows/matchers/pylint.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/matchers/pylint.json b/.github/workflows/matchers/pylint.json index 5624ca695c..d796afb259 100644 --- a/.github/workflows/matchers/pylint.json +++ b/.github/workflows/matchers/pylint.json @@ -19,7 +19,7 @@ "severity": "warning", "pattern": [ { - "regexp": "^(.+):(\\d+):(\\d+):\\s(([CRW]\\d{4}):\\s.+)$", + "regexp": "^(.+):(\\d+):(\\d+):\\s(([CRWI]\\d{4}):\\s.+)$", "file": 1, "line": 2, "column": 3, From 8eb15d555af0d74b84dd36a853c99e6a45276214 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 3 May 2023 23:47:58 +0000 Subject: [PATCH 007/155] docs: collapse install variants into per-hardware sections in README --- README.md | 142 +++++++++++++++++++++++------------------------------- 1 file changed, 60 insertions(+), 82 deletions(-) diff --git a/README.md b/README.md index c0e5e6cfc8..6020570586 100644 --- a/README.md +++ b/README.md @@ -13,17 +13,7 @@ - [📋 Requirements](#-requirements) - [✅ Getting started](#-getting-started) - [🧰 Installing `ilab`](#-installing-ilab) - - [To install with no GPU acceleration and PyTorch without CUDA bindings](#to-install-with-no-gpu-acceleration-and-pytorch-without-cuda-bindings) - - [To install with AMD ROCm](#to-install-with-amd-rocm) - - [To install with Apple Metal on M1/M2/M3 Mac](#to-install-with-apple-metal-on-m1m2m3-mac) - - [To install with Nvidia CUDA](#to-install-with-nvidia-cuda) - - [Example output](#example-output) - - [Bash (version 4.4 or newer)](#bash-version-44-or-newer) - - [Zsh](#zsh) - - [Fish](#fish) - [🏗️ Initialize `ilab`](#️-initialize-ilab) - - [Example output](#example-output-1) - - [Example output](#example-output-2) - [📥 Download the model](#-download-the-model) - [🍴 Serving the model](#-serving-the-model) - [📣 Chat with the model (Optional)](#-chat-with-the-model-optional) @@ -31,12 +21,7 @@ - [🎁 Contribute knowledge or compositional skills](#-contribute-knowledge-or-compositional-skills) - [📜 List and validate your new data](#-list-and-validate-your-new-data) - [🚀 Generate a synthetic dataset](#-generate-a-synthetic-dataset) - - [Example output](#example-output-3) - [👩‍🏫 Train the model](#-train-the-model) - - [Train the model locally on Linux](#train-the-model-locally-on-linux) - - [Train the model locally on an M-series Mac](#train-the-model-locally-on-an-m-series-mac) - - [Training the model locally with GPU acceleration](#training-the-model-locally-with-gpu-acceleration) - - [Training the model in the cloud](#training-the-model-in-the-cloud) - [📜 Test the newly trained model](#-test-the-newly-trained-model) - [🍴 Serve the newly trained model](#-serve-the-newly-trained-model) - [📣 Chat with the new model (not optional this time)](#-chat-with-the-new-model-not-optional-this-time) @@ -56,15 +41,15 @@ Large Language Models (LLMs.) The "**lab**" in Instruct**Lab** 🐶 stands for `ilab` is a Command-Line Interface (CLI) tool that allows you to perform the following actions: 1. Download a pre-trained Large Language Model (LLM). -1. Chat with the LLM. +2. Chat with the LLM. To add new knowledge and skills to the pre-trained LLM, add information to the companion [taxonomy](https://github.com/instructlab/taxonomy.git) repository. After you have added knowledge and skills to the taxonomy, you can perform the following actions: 1. Use `ilab` to generate new synthetic training data based on the changes in your local `taxonomy` repository. -1. Re-train the LLM with the new training data. -1. Chat with the re-trained LLM to see the results. +2. Re-train the LLM with the new training data. +3. Chat with the re-trained LLM to see the results. ```mermaid graph TD; @@ -107,7 +92,7 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow sudo dnf install gcc-c++ gcc make pip python3 python3-devel python3-GitPython ``` - If you are running on macOS, this installation is not necessary and you can begin your process with the following step. + If you are running on macOS, this installation is not necessary and you can begin your process with the following step. 2. Create a new directory called `instructlab` to store the files the `ilab` CLI needs when running and `cd` into the directory by running the following command: @@ -123,10 +108,15 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow > **NOTE**: ⏳ `pip install` may take some time, depending on your internet connection. In case installation fails with error ``unsupported instruction `vpdpbusd'``, append `-C cmake.args="-DLLAMA_NATIVE=off"` to `pip install` command. See [the GPU acceleration documentation](./docs/gpu-acceleration.md) for how to - to enable hardware acceleration for inference and training on AMD ROCm, + enable hardware acceleration for inference and training on AMD ROCm, Apple Metal Performance Shaders (MPS), and Nvidia CUDA. - #### To install with no GPU acceleration and PyTorch without CUDA bindings + Pick the installation matching your hardware below. All variants follow the same three steps: + create a virtual env, activate it, clear any cached `llama_cpp_python` wheel, and `pip install` + `ilab` with the appropriate extra-index-url or cmake args. + +
+ CPU only (no GPU acceleration) ```shell python3 -m venv --upgrade-deps venv @@ -135,7 +125,10 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow (venv) $ pip install git+https://github.com/instructlab/instructlab.git@stable --extra-index-url=https://download.pytorch.org/whl/cpu ``` - #### To install with AMD ROCm +
+ +
+ AMD ROCm ```shell python3 -m venv --upgrade-deps venv @@ -152,7 +145,10 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow On Fedora 40+, use `-DCMAKE_C_COMPILER=clang-17` and `-DCMAKE_CXX_COMPILER=clang++-17`. - #### To install with Apple Metal on M1/M2/M3 Mac +
+ +
+ Apple Metal (M1/M2/M3) ```shell python3 -m venv --upgrade-deps venv @@ -161,7 +157,10 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow (venv) $ pip install git+https://github.com/instructlab/instructlab.git@stable -C cmake.args="-DLLAMA_METAL=on" ``` - #### To install with Nvidia CUDA +
+ +
+ Nvidia CUDA ```shell python3 -m venv --upgrade-deps venv @@ -170,13 +169,15 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow (venv) $ pip install git+https://github.com/instructlab/instructlab.git@stable -C cmake.args="-DLLAMA_CUBLAS=on" ``` -4. From your `venv` environment, verify `ilab` is installed correctly, by running the `ilab` command. +
+ +4. From your `venv` environment, verify `ilab` is installed correctly by running the `ilab` command. ```shell ilab ``` - #### Example output + Example output: ```shell (venv) $ ilab @@ -211,52 +212,33 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow source venv/bin/activate ``` -5. You may optionally enable tab completion for the `ilab` command. - - #### Bash (version 4.4 or newer) +5. You may optionally enable tab completion for the `ilab` command. Pick the snippet for your + shell; the one-liner enables completion for the current session, while the two-liner persists + it across new shells. - Enable tab completion in `bash` with the following command: + **Bash (4.4+):** ```sh eval "$(_ILAB_COMPLETE=bash_source ilab)" - ``` - - To have this enabled automatically every time you open a new shell, - you can save the completion script and source it from `~/.bashrc`: - - ```sh + # persist: _ILAB_COMPLETE=bash_source ilab > ~/.ilab-complete.bash echo ". ~/.ilab-complete.bash" >> ~/.bashrc ``` - #### Zsh - - Enable tab completion in `zsh` with the following command: + **Zsh:** ```sh eval "$(_ILAB_COMPLETE=zsh_source ilab)" - ``` - - To have this enabled automatically every time you open a new shell, - you can save the completion script and source it from `~/.zshrc`: - - ```sh + # persist: _ILAB_COMPLETE=zsh_source ilab > ~/.ilab-complete.zsh echo ". ~/.ilab-complete.zsh" >> ~/.zshrc ``` - #### Fish - - Enable tab completion in `fish` with the following command: + **Fish:** ```sh _ILAB_COMPLETE=fish_source ilab | source - ``` - - To have this enabled automatically every time you open a new shell, - you can save the completion script and source it from `~/.bashrc`: - - ```sh + # persist: _ILAB_COMPLETE=fish_source ilab > ~/.config/fish/completions/ilab.fish ``` @@ -268,7 +250,7 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow ilab init ``` - #### Example output + Example output: ```shell Welcome to InstructLab CLI. This guide will help you set up your environment. @@ -282,7 +264,7 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow **Optional**: If you want to point to an existing local clone of the `taxonomy` repository, you can pass the path interactively or alternatively with the `--taxonomy-path` flag. - #### Example output + Example output of a full run: ```shell (venv) $ ilab init @@ -309,7 +291,7 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow ### 📥 Download the model -- Run the `ilab download`command. +- Run the `ilab download` command. ```shell ilab download @@ -324,7 +306,7 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow merlinite-7b-lab-Q4_K_M.gguf ``` - > **NOTE** ⏳ This command can take few minutes or immediately depending on your internet connection or model is cached. If you have issues connecting to Hugging Face, refer to the [Hugging Face discussion forum](https://discuss.huggingface.co/) for more details. + > **NOTE** ⏳ This command can take a few minutes or complete immediately depending on your internet connection or whether the model is cached. If you have issues connecting to Hugging Face, refer to the [Hugging Face discussion forum](https://discuss.huggingface.co/) for more details. ### 🍴 Serving the model @@ -348,7 +330,7 @@ For an overview of the full workflow, see the [workflow diagram](./docs/workflow ### 📣 Chat with the model (Optional) -Because you're serving the model in one terminal window, you will have to create a new window and re-activate your Python virtual environment to run `ilab chat` command: +Because you're serving the model in one terminal window, you will have to create a new window and re-activate your Python virtual environment to run the `ilab chat` command: ```shell source venv/bin/activate @@ -361,15 +343,12 @@ Before you start adding new skills and knowledge to your model, you can check it ```shell (venv) $ ilab chat -╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────── system ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ -│ Welcome to InstructLab Chat w/ GGML-MERLINITE-7B-lab-Q4_K_M (type /h for help) │ -╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ->>b> what is the capital of Canada [S][default] -╭────────────────────────────────────────────────────────────────────────────────────────────────────── ggml-merlinite-7b-lab-Q4_K_M ───────────────────────────────────────────────────────────────────────────────────────────────────────╮ -│ The capital city of Canada is Ottawa. It is located in the province of Ontario, on the southern banks of the Ottawa River in the eastern portion of southern Ontario. The city serves as the political center for Canada, as it is home to │ -│ Parliament Hill, which houses the House of Commons, Senate, Supreme Court, and Cabinet of Canada. Ottawa has a rich history and cultural significance, making it an essential part of Canada's identity. │ -╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── elapsed 12.008 seconds ─╯ ->>> [S][default] +>>> what is the capital of Canada +The capital city of Canada is Ottawa. It is located in the province of Ontario, +on the southern banks of the Ottawa River in the eastern portion of southern +Ontario. The city serves as the political center for Canada, as it is home to +Parliament Hill, which houses the House of Commons, Senate, Supreme Court, and +Cabinet of Canada. ``` ## 💻 Creating new knowledge or skills and training the model @@ -403,7 +382,7 @@ Detailed contribution instructions can be found in the [taxonomy repository](htt Before following these instructions, ensure the existing model you are adding skills or knowledge to is still running. -1. To generate a synthetic dataset based on your newly added knowledge or skill set in [taxonomy](https://github.com/instructlab/taxonomy.git) repository, run the following command: +1. To generate a synthetic dataset based on your newly added knowledge or skill set in the [taxonomy](https://github.com/instructlab/taxonomy.git) repository, run the following command: ```shell ilab generate @@ -411,7 +390,7 @@ Before following these instructions, ensure the existing model you are adding sk > **NOTE:** ⏳ This can take from 15 minutes to 1+ hours to complete, depending on your computing resources. - #### Example output + Example output: ```shell (venv) $ ilab generate @@ -425,8 +404,8 @@ Before following these instructions, ensure the existing model you are adding sk The synthetic data set will be three files in the newly created `generated` directory named `generated*.json`, `test*.jsonl`, and `train*.jsonl`. > [!NOTE] -> If you want to pickup from where a failed or canceled `ilab generate` left off, you can copy the -> `generated*.json` file into a file named `regen.json`. `regen.json` will be picked up at the start of `lab +> If you want to pick up from where a failed or canceled `ilab generate` left off, you can copy the +> `generated*.json` file into a file named `regen.json`. `regen.json` will be picked up at the start of `ilab > generate` when available. You should remove it when the process is completed. 2. Verify the files have been created by running the `ls generated` command. @@ -467,7 +446,7 @@ ilab train #### Train the model locally on an M-series Mac -To train the model locally on your M-Series Mac is as easy as running: +To train the model locally on your M-series Mac, run: ```shell ilab train @@ -483,7 +462,7 @@ and output of `ilab generate` but on the order of 5 to 15 minutes) adapters-010.npz adapters-050.npz adapters-090.npz config.json tokenizer.model adapters-020.npz adapters-060.npz adapters-100.npz model.safetensors tokenizer_config.json adapters-030.npz adapters-070.npz adapters.npz special_tokens_map.json -adapters-040.npz adapters-080.npz added_tokens.json tokenizer.jso +adapters-040.npz adapters-080.npz added_tokens.json tokenizer.json ``` #### Training the model locally with GPU acceleration @@ -501,9 +480,9 @@ Follow the instructions in [Training](./notebooks/README.md). ⏳ Approximate amount of time taken on each platform: - *Google Colab*: **5-10 minutes** with a T4 GPU -- *Kaggle*: **~30 minutes** with a P100 GPU. +- *Kaggle*: **~30 minutes** with a P100 GPU -After that's done, you can play with your model directly in the Google Colab or Kaggle notebook. Model trained on the cloud will be saved on the cloud. +After that's done, you can play with your model directly in the Google Colab or Kaggle notebook. A model trained in the cloud will be saved in the cloud. The model can also be downloaded and served locally. ### 📜 Test the newly trained model @@ -523,10 +502,9 @@ The model can also be downloaded and served locally. 1. Stop the server you have running by entering `ctrl+c` keys in the terminal running the server. > **IMPORTANT**: - - 🍎 This step is only implemented for macOS with M-series chips (for now). - - - Before serving the newly trained model you must convert it to work with - the `ilab` cli. The `ilab convert` command converts the new model into quantized [GGUF](https://medium.com/@sandyeep70/ggml-to-gguf-a-leap-in-language-model-file-formats-cd5d3a6058f9) format which is required by the server to host the model in the `ilab serve` command. + > - 🍎 This step is only implemented for macOS with M-series chips (for now). + > - Before serving the newly trained model you must convert it to work with + > the `ilab` CLI. The `ilab convert` command converts the new model into quantized [GGUF](https://medium.com/@sandyeep70/ggml-to-gguf-a-leap-in-language-model-file-formats-cd5d3a6058f9) format which is required by the server to host the model in the `ilab serve` command. 2. Convert the newly trained model by running the following command: @@ -534,8 +512,8 @@ The model can also be downloaded and served locally. ilab convert ``` -3. Serve the newly trained model locally via `ilab serve` command with the `--model-path` -argument to specify your new model: +3. Serve the newly trained model locally via the `ilab serve` command with the `--model-path` + argument to specify your new model: ```shell ilab serve --model-path From e2d0fd8d489ca4302bdaec4e91031556dae7ff3a Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 5 May 2023 18:28:27 +0000 Subject: [PATCH 008/155] docs(rocm): note group re-login requirement and GPU verify step --- containers/rocm/README.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/containers/rocm/README.md b/containers/rocm/README.md index cffa364691..d9c85d5bcc 100644 --- a/containers/rocm/README.md +++ b/containers/rocm/README.md @@ -17,12 +17,21 @@ The container has all Python dependencies installed in a virtual env. The virtua 6. enter toolbox `toolbox enter instructlab`. The container has your home directory mounted. +Note: after adding yourself to the `render` and `video` groups, you must log +out and back in (or run `newgrp`) for the new group membership to take effect. +Without that, the container will not have access to `/dev/kfd` and GPU +inference will silently fall back to CPU, which is dramatically slower. + To update InstructLab CLI to latest version: `pip install -e ~/path/to/instructlab/instructlab` `ilab generate` and `ilab chat` use the GPU automatically. `ilab train` needs more powerful and recent GPU and therefore does not use GPU by default. To train on a GPU, run `ilab train --device cuda`. +You can verify that the GPU is visible inside the toolbox by running +`rocminfo | grep -E 'Name|gfx'` and confirming that your card's `gfx` target +is listed as an agent. + ## Building for other GPU architectures Use the `amdgpu-arch` or `rocminfo` tool to get the short name @@ -68,4 +77,4 @@ make rocm-gfx1100 BUILD_ARGS= ## Known issues -AMD Instinct MI210 with ISA `amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-` is not supported by Fedora build `rocblas-6.0.2-3`. As of late April 2024, Fedora has `gfx90a:xnack+` and `gfx90a:xnack-` but lacks `gfx90a:sramecc+:xnack-`. \ No newline at end of file +AMD Instinct MI210 with ISA `amdgcn-amd-amdhsa--gfx90a:sramecc+:xnack-` is not supported by Fedora build `rocblas-6.0.2-3`. As of late April 2024, Fedora has `gfx90a:xnack+` and `gfx90a:xnack-` but lacks `gfx90a:sramecc+:xnack-`. From 91c159aa290006ea53bf53fa9d13588139efee48 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sat, 6 May 2023 21:52:41 +0000 Subject: [PATCH 009/155] docs: fix typo in CLI repo link in demo slides --- docs/demo-slides.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/demo-slides.md b/docs/demo-slides.md index f53a784d7c..b4398c545d 100644 --- a/docs/demo-slides.md +++ b/docs/demo-slides.md @@ -192,4 +192,4 @@ git push --set-upstream xukai92 demo --> [instructlab/taxonomy](https://github.com/instructlab/taxonomy) -For more info, see [the CLI repo](https://github.com/instructlab/instuctlab). +For more info, see [the CLI repo](https://github.com/instructlab/instructlab). From 86b468d3bde71c90bf27701425374119a34117f7 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sun, 7 May 2023 17:01:41 +0000 Subject: [PATCH 010/155] Expand bug report template with logs, accelerator, and config sections --- .github/ISSUE_TEMPLATE/bug_report.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index b3fbf68048..e912e092cf 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -17,17 +17,28 @@ Steps to reproduce the behavior: 3. Scroll down to '....' 4. See error +If possible, please include the exact command(s) you ran and the full +output (use code blocks for readability). + **Expected behavior** -**Screenshots** - +**Screenshots or Logs** + **Device Info (please complete the following information):** - Hardware Specs: [e.g. Apple M2 Pro Chip, 16 GB Memory, etc.] - OS Version: [e.g. Mac OS 14.4.1, Fedora Linux 40] - Python Version: [output of `python --version`] - InstructLab Version: [output of `ilab --version`] + - Accelerator / GPU (if applicable): [e.g. NVIDIA RTX 4090, AMD MI300X, none] + +**Configuration** + **Additional context** - + From 801f0b62d1d2fd929b543c3f30cc54c8644c7fd6 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 10 May 2023 15:22:51 +0000 Subject: [PATCH 011/155] Add alternatives section to feature request template --- .github/ISSUE_TEMPLATE/feature_request.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 37800abd86..6315abc659 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -13,5 +13,8 @@ assignees: '' **Describe the solution you'd like** +**Describe alternatives you've considered** + + **Additional context** From 0c1665aa2000ae63b9ccc616c43df2e1c27c5068 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 11 May 2023 14:28:46 +0000 Subject: [PATCH 012/155] docs: clarify release branch naming and add Y-stream definition --- docs/release-strategy.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/release-strategy.md b/docs/release-strategy.md index d0d1d5e742..4dfdcd9340 100644 --- a/docs/release-strategy.md +++ b/docs/release-strategy.md @@ -8,7 +8,9 @@ This document discusses the release strategy and processes for the Releases use a `X.Y.Z` numbering scheme. -X-stream release are for major releases. At this stage in the project a major release has not been cut and we expect each release to be a new Y-stream. +X-stream releases are for major releases. At this stage in the project a major release has not been cut and we expect each release to be a new Y-stream. + +Y-stream releases are for feature releases that may include new functionality, dependency updates, and non-critical bug fixes. Z-stream releases are meant for critical bug and documentation fixes. Z-stream releases are cut as maintainers see fit. @@ -29,14 +31,14 @@ PRs and Issues associated with the next two milestones will be prioritized for r ## Git Branches and Tags -Every `X.Y` release stream gets a new branch. +Every `X.Y` release stream gets a new branch named `release-vX.Y`. Each release, `X.Y.Z`, exists as a tag named `vX.Y.Z`. ## Release Branch Maintenance Maintenance efforts are only on the most recent Y-stream. -Critical bug fixes are backported to the most recent `X.Y` branch that contains the `stable` tag for a Z-stream release. +Critical bug fixes are backported to the most recent `release-vX.Y` branch that contains the `stable` tag for a Z-stream release. ## Release Mechanics @@ -45,13 +47,13 @@ The Release Manager is a member of the CLI Maintainers team that has agreed to t The Release Manager can change on a per-release basis. The Release Manager for each release is identified in the Description of the Milestone used to plan and track that release on GitHub. -The following are the steps for how Y-stream and Z-stream releases gets cut. +The following are the steps for how Y-stream and Z-stream releases get cut. 1. Maintainers determine a commit on the main branch that will serve as the basis for the next release. - For a Z-stream release skip this step. 2. For a Y-Stream release: create a new branch in the format `release-vX.Y`. - For a Z-Stream release skip this step. -3. For a Z-Stream release: backport all relevant commits from `main` to the `release-X.Y` branch. +3. For a Z-Stream release: backport all relevant commits from `main` to the `release-vX.Y` branch. - For a Y-Stream release skip this step. 4. Create a new release on GitHub. The following is automated: - Tagging the branch on GitHub From fb43a34c44192d61267a8d468da173d23d9deb3b Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sun, 14 May 2023 21:34:42 +0000 Subject: [PATCH 013/155] Expand skill-wiki.md with categories, see-also, and refs --- tests/testdata/temp_repo/docs/skill-wiki.md | 28 ++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/tests/testdata/temp_repo/docs/skill-wiki.md b/tests/testdata/temp_repo/docs/skill-wiki.md index ce53550a71..86c628309f 100644 --- a/tests/testdata/temp_repo/docs/skill-wiki.md +++ b/tests/testdata/temp_repo/docs/skill-wiki.md @@ -3,5 +3,31 @@ # Skill A skill is the learned ability to act with determined results with good execution often within a given amount of time, energy, or both. Skills can often be divided into domain-general and domain-specific skills. For example, in the domain of work, some general skills would include time management, teamwork and leadership, self-motivation and others, whereas domain-specific skills would be used only for a certain job. Skill usually requires certain environmental stimuli and situations to assess the level of skill being shown and used.[citation needed] + A skill may be called an art when it represents a body of knowledge or branch of learning, as in the art of medicine or the art of war.[1] Although the arts are also skills, there are many skills that form an art but have no connection to the fine arts. -People need a broad range of skills to contribute to the modern economy. A U.S. Department of Labor study showed that through technology, the workplace is changing, and identified 16 basic skills that employees must have to be able to change with it.[2] Three broad categories of skills are suggested and these are technical, human, and conceptual.[3] The first two can be substituted with hard and soft skills, respectively.[4] \ No newline at end of file + +People need a broad range of skills to contribute to the modern economy. A U.S. Department of Labor study showed that through technology, the workplace is changing, and identified 16 basic skills that employees must have to be able to change with it.[2] Three broad categories of skills are suggested and these are technical, human, and conceptual.[3] The first two can be substituted with hard and soft skills, respectively.[4] + +## Categories of skills + +- **Technical skills**: Job-specific abilities such as operating machinery, writing software, or performing surgery. +- **Human skills**: Interpersonal abilities such as communication, empathy, and collaboration. +- **Conceptual skills**: Higher-order abilities such as analysis, synthesis, and strategic thinking. + +## Hard vs. soft skills + +Hard skills are typically teachable and measurable (e.g., typing speed, fluency in a programming language), while soft skills relate to character traits and interpersonal effectiveness (e.g., adaptability, negotiation, time management). + +## See also + +- Competence (human resources) +- Expertise +- Learning +- Training + +## References + +1. *Oxford English Dictionary*, entry for "art". +2. U.S. Department of Labor, Secretary's Commission on Achieving Necessary Skills (SCANS) report. +3. Katz, R. L. (1955). "Skills of an Effective Administrator". *Harvard Business Review*. +4. Laker, D. R.; Powell, J. L. (2011). "The differences between hard and soft skills and their relative impact on training transfer". *Human Resource Development Quarterly*. From 55af3294cf4634e9533209ffe834fcb2b81fb69f Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 17 May 2023 14:32:42 +0000 Subject: [PATCH 014/155] Clarify reporting guidance in Code of Conduct --- CODE_OF_CONDUCT.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 14cd1f2473..5417ef9b18 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -1,3 +1,5 @@ # InstructLab/cli - Code of Conduct and Covenant -This project adheres to the [InstructLab - Code of Conduct and Covenant](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. +This project adheres to the [InstructLab - Code of Conduct and Covenant](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md). By participating in this project, you are expected to uphold this code. + +Please report any unacceptable behavior in accordance with the reporting guidelines outlined in the linked Code of Conduct. From 99604db8f6b19a08b9f53588ea0f693c7739efc1 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 18 May 2023 16:15:29 +0000 Subject: [PATCH 015/155] docs: clarify governance link references community repo --- governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/governance.md b/governance.md index 1308193c18..a52fec09d2 100644 --- a/governance.md +++ b/governance.md @@ -1,3 +1,3 @@ # InstructLab Governance -For information about how the InstructLab project governance operates, see [InstructLab Governance](https://github.com/instructlab/community/blob/main/governance.md#instructlab-governance). \ No newline at end of file +For information about how the InstructLab project governance operates, see the [InstructLab Governance](https://github.com/instructlab/community/blob/main/governance.md#instructlab-governance) document in the `instructlab/community` repository. From a206f8afaffd6f7be89d5032f4f926fbe185a323 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 19 May 2023 16:52:15 +0000 Subject: [PATCH 016/155] docs: expand PlantUML guide with local server and editing tips --- docs/README.md | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-) diff --git a/docs/README.md b/docs/README.md index 8f65baecd6..80d1906277 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,6 +6,8 @@ To generate it yourself, the easiest way is to install the [PlantUML plugin in VS Code](https://marketplace.visualstudio.com/items?itemName=jebbs.plantuml) (with the prerequisite installed), open the file and click preview. +## Remote rendering + If you don't want to install the dependencies locally, you can use the following settings to make the preview work with a remote render: @@ -14,4 +16,25 @@ settings to make the preview work with a remote render: "plantuml.server": "https://www.plantuml.com/plantuml", ``` -[ASCIIFlow](https://asciiflow.com/#/) is a helpful tool to edit the source code. +Note that the public PlantUML server may be slow or rate-limited during peak +hours. For faster iteration, consider running a local server via Docker: + +```bash +docker run -d -p 8080:8080 plantuml/plantuml-server:jetty +``` + +Then point the `plantuml.server` setting to `http://localhost:8080`. + +## Editing tips + +[ASCIIFlow](https://asciiflow.com/#/) is a helpful tool to edit the source code +of the ditaa diagrams. When the diagram grows large, prefer splitting it into +multiple smaller figures rather than a single dense one — rendering time scales +roughly with the number of cells, and smaller diagrams are easier to review in +pull requests. + +## Regenerating figures + +After editing a `.puml` source, export the rendered PNG/SVG alongside the +source file and commit both. This avoids forcing every reader to re-run the +renderer locally. From 965ed9797604c33583f52490dc07863115f49be3 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 30 May 2023 13:54:42 +0000 Subject: [PATCH 017/155] Add checklist section to PR template --- .github/PULL_REQUEST_TEMPLATE.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 259249fd0b..520a816461 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -4,3 +4,7 @@ Resolves # **Description of your changes:** + +**Checklist:** +- [ ] Tests added or updated +- [ ] Documentation updated (if applicable) From 55a73c40bda6757e1825646c773cdbd7f33608a6 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 1 Jun 2023 17:19:30 +0000 Subject: [PATCH 018/155] docs(llamacpp): document modification policy in NOTICE --- src/instructlab/llamacpp/README.md | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/src/instructlab/llamacpp/README.md b/src/instructlab/llamacpp/README.md index b350776777..14ee87654d 100644 --- a/src/instructlab/llamacpp/README.md +++ b/src/instructlab/llamacpp/README.md @@ -1,8 +1,22 @@ # Notice -The code in this folder is modified from the llama.cpp repository on GitHub. -For the original code, see [llama.cpp](https://github.com/ggerganov/llama.cpp) -with the following license. +The code in this folder is modified from the [llama.cpp](https://github.com/ggerganov/llama.cpp) +repository on GitHub. The original code is distributed under the MIT License, +reproduced below. + +## Modifications + +Local modifications are limited to integration glue required by +`instructlab` and do not alter the core licensing terms of the upstream +project. When updating files in this directory, please: + +1. Preserve this `NOTICE` and the original MIT License text below. +2. Record the upstream commit or release that the change is based on in + the corresponding pull request description. +3. Keep modifications minimal and clearly separated from upstream code + where practical. + +## Original License ```license MIT License From 906d3c004b37b6abb39844e8dce7049c778686e3 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sun, 4 Jun 2023 19:16:22 +0000 Subject: [PATCH 019/155] docs: advise against bundling unrelated refactors in llama.cpp updates --- src/instructlab/llamacpp/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/instructlab/llamacpp/README.md b/src/instructlab/llamacpp/README.md index 14ee87654d..05390effb1 100644 --- a/src/instructlab/llamacpp/README.md +++ b/src/instructlab/llamacpp/README.md @@ -15,6 +15,8 @@ project. When updating files in this directory, please: the corresponding pull request description. 3. Keep modifications minimal and clearly separated from upstream code where practical. +4. Avoid bundling unrelated refactors in the same change so the diff + against upstream remains easy to audit. ## Original License From 0a4748ed2436cb6db3704abccd732aa71be510f3 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 5 Jun 2023 15:34:43 +0000 Subject: [PATCH 020/155] docs: expand Code of Conduct with reporting and scope sections --- CODE_OF_CONDUCT.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 5417ef9b18..3c21989f1f 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -2,4 +2,19 @@ This project adheres to the [InstructLab - Code of Conduct and Covenant](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md). By participating in this project, you are expected to uphold this code. +## Reporting + Please report any unacceptable behavior in accordance with the reporting guidelines outlined in the linked Code of Conduct. + +When reporting an issue, it is helpful to include: + +- A description of what happened, including any relevant context. +- The approximate date, time, and location (e.g., GitHub issue, pull request, chat channel) of the incident. +- The names or handles of any individuals involved, if known. +- Links to any public records of the behavior (e.g., comments, commits, messages), if available. + +Reports are handled confidentially by the InstructLab community maintainers. See the linked Code of Conduct for the current list of contacts and the full reporting process. + +## Scope + +This Code of Conduct applies within all project spaces — including the code repository, issue tracker, pull requests, discussions, and any official communication channels — as well as when an individual is officially representing the project in public spaces. From aeb96d67b2e3b9c00df8b23c0e39671ef25eefc6 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 16 Jun 2023 01:26:22 +0000 Subject: [PATCH 021/155] docs(habana): indent step code blocks and use sudo for dnf/modprobe --- docs/habana-gaudi.md | 102 +++++++++++++++++++++---------------------- 1 file changed, 51 insertions(+), 51 deletions(-) diff --git a/docs/habana-gaudi.md b/docs/habana-gaudi.md index bcb5ef2a49..2d22eb005d 100644 --- a/docs/habana-gaudi.md +++ b/docs/habana-gaudi.md @@ -18,73 +18,73 @@ 1. Enable CRB and EPEL repositories -```shell -sudo subscription-manager repos --enable codeready-builder-for-rhel-9-$(arch)-rpms -sudo dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm -``` + ```shell + sudo subscription-manager repos --enable codeready-builder-for-rhel-9-$(arch)-rpms + sudo dnf install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm + ``` 2. Add Habana Vault repository `/etc/yum.repos.d/Habana-Vault.repo` -```ini -[vault] -name=Habana Vault -baseurl=https://vault.habana.ai/artifactory/rhel/9/9.2 -enabled=1 -repo_gpgcheck=0 -``` + ```ini + [vault] + name=Habana Vault + baseurl=https://vault.habana.ai/artifactory/rhel/9/9.2 + enabled=1 + repo_gpgcheck=0 + ``` 3. Install firmware and tools -```shell -dnf install habanalabs-firmware habanalabs-firmware-tools -``` + ```shell + sudo dnf install habanalabs-firmware habanalabs-firmware-tools + ``` 4. Install Kernel drivers. This will build and install several Kernel modules with DKMS -```shell -dnf install habanalabs -``` + ```shell + sudo dnf install habanalabs + ``` 5. Load Kernel drivers -```shell -modprobe habanalabs_en habanalabs_cn habanalabs -``` + ```shell + sudo modprobe habanalabs_en habanalabs_cn habanalabs + ``` 6. Check journald for device -```shell -journalctl -o cat | grep habanalabs -habanalabs hl0: Loading secured firmware to device, may take some time... -habanalabs hl0: preboot full version: 'Preboot version hl-gaudi2-1.14.0-fw-48.0.1-sec-7 (Jan 07 2024 - 20:03:16)' -habanalabs hl0: boot-fit version 49.0.0-sec-9 -habanalabs hl0: Successfully loaded firmware to device -habanalabs hl0: Linux version 49.0.0-sec-9 -habanalabs hl0: Found GAUDI2 device with 96GB DRAM -habanalabs hl0: hwmon1: add sensors information -habanalabs hl0: Successfully added device 0000:19:00.0 to habanalabs driver -``` + ```shell + journalctl -o cat | grep habanalabs + habanalabs hl0: Loading secured firmware to device, may take some time... + habanalabs hl0: preboot full version: 'Preboot version hl-gaudi2-1.14.0-fw-48.0.1-sec-7 (Jan 07 2024 - 20:03:16)' + habanalabs hl0: boot-fit version 49.0.0-sec-9 + habanalabs hl0: Successfully loaded firmware to device + habanalabs hl0: Linux version 49.0.0-sec-9 + habanalabs hl0: Found GAUDI2 device with 96GB DRAM + habanalabs hl0: hwmon1: add sensors information + habanalabs hl0: Successfully added device 0000:19:00.0 to habanalabs driver + ``` 7. Check `hl-smi` -````shell -hl-smi -+-----------------------------------------------------------------------------+ -| HL-SMI Version: hl-1.15.1-fw-49.0.0.0 | -| Driver Version: 1.15.1-62f612b | -|-------------------------------+----------------------+----------------------+ -| AIP Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | -| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | AIP-Util Compute M. | -|===============================+======================+======================| -| 0 HL-225 N/A | 0000:19:00.0 N/A | 0 | -| N/A 29C N/A 93W / 600W | 768MiB / 98304MiB | 0% N/A | -|-------------------------------+----------------------+----------------------+ -| Compute Processes: AIP Memory | -| AIP PID Type Process name Usage | -|=============================================================================| -| 0 N/A N/A N/A N/A | -+=============================================================================+ -```` + ````shell + hl-smi + +-----------------------------------------------------------------------------+ + | HL-SMI Version: hl-1.15.1-fw-49.0.0.0 | + | Driver Version: 1.15.1-62f612b | + |-------------------------------+----------------------+----------------------+ + | AIP Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | + | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | AIP-Util Compute M. | + |===============================+======================+======================| + | 0 HL-225 N/A | 0000:19:00.0 N/A | 0 | + | N/A 29C N/A 93W / 600W | 768MiB / 98304MiB | 0% N/A | + |-------------------------------+----------------------+----------------------+ + | Compute Processes: AIP Memory | + | AIP PID Type Process name Usage | + |=============================================================================| + | 0 N/A N/A N/A N/A | + +=============================================================================+ + ```` See [Intel Gaudi SW Stack for RHEL 9.2](https://docs.habana.ai/en/latest/shared/Install_Driver_and_Firmware.html) for detailed documentation. @@ -94,7 +94,7 @@ for detailed documentation. The Habana Vault repository provides several other tools, e.g. OCI container runtime hooks. ```shell -dnf install habanalabs-graph habanatool habanalabs-thunk habanalabs-container-runtime +sudo dnf install habanalabs-graph habanatool habanalabs-thunk habanalabs-container-runtime ``` ## Install Python, Intel oneMKL, and PyTorch stack @@ -213,7 +213,7 @@ PT and Habana Environment variables ## Container ```shell -dnf install habanalabs-container-runtime podman +sudo dnf install habanalabs-container-runtime podman make hpu podman run -ti --privileged -v ./data:/opt/app-root/src:z localhost/instructlab:hpu ``` From 695c0e5aec9b297901a24209bf992b3cd0da4ac9 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 6 Jul 2023 00:25:54 +0000 Subject: [PATCH 022/155] docs(ci): expand E2E docs with inputs, CLI usage, a11y notes --- docs/ci.md | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 63 insertions(+), 3 deletions(-) diff --git a/docs/ci.md b/docs/ci.md index cec529048a..6824697680 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -1,10 +1,16 @@ # CI for InstructLab +This document describes the continuous integration (CI) jobs used by the +`instructlab` project, how to trigger them, and what they cover. + ## End-to-end CI Job -This CI job is manually triggered by `instructlab` repo maintainers. It runs as -much of the `ilab` workflow as it can on the GPU-enabled worker we have -available through GitHub Actions. +The end-to-end (E2E) job exercises as much of the `ilab` workflow as possible +on a GPU-enabled GitHub Actions runner. Because GPU minutes are a limited +resource, this job is **manually triggered** by `instructlab` repository +maintainers rather than running on every pull request. + +### Triggering the workflow 1. Visit the [Actions tab](https://github.com/instructlab/instructlab/actions). 2. Click on the [E2E test](https://github.com/instructlab/instructlab/actions/workflows/e2e.yml) @@ -12,3 +18,57 @@ available through GitHub Actions. 3. Click on the `Run workflow` button on the right side of the page. 4. Enter a branch name or a PR number in the input field. 5. Click the green `Run workflow` button. + +### Inputs + +| Input | Description | +| ------ | ---------------------------------------------------------- | +| `pr_or_branch` | A PR number (e.g. `1234`) or a branch name to test. | + +### What the job runs + +At a high level, the E2E job performs the following steps: + +- Checks out the requested branch or PR. +- Installs `ilab` and its dependencies on a GPU-enabled runner. +- Initializes a working directory with `ilab config init`. +- Downloads a model and runs a short generation to validate inference. +- Runs a minimal synthetic data generation and training pass. +- Uploads logs as workflow artifacts for later inspection. + +If any step fails, the job aborts and the logs from the failing step are +available in the run summary. + +## Accessibility + +The steps above rely on the GitHub Actions web UI. To make this workflow +usable for contributors who navigate with assistive technology, keep the +following in mind: + +- All buttons referenced (`Run workflow`) are standard GitHub UI elements + with accessible names and can be activated with the keyboard (`Tab` to + focus, `Enter` or `Space` to activate). +- Screen reader users can jump to the workflow list using the page's + landmark navigation; the workflows are rendered as a list of links. +- If you cannot use the web UI, the same workflow can be dispatched from + the command line using the [GitHub CLI](https://cli.github.com/): + + ```shell + gh workflow run e2e.yml -f pr_or_branch= + ``` + + You can then stream the run's logs in your terminal with: + + ```shell + gh run watch + ``` + +Please open an issue if any part of this process is not accessible to you; +we treat accessibility bugs as regular bugs. + +## Reporting CI issues + +If a CI job appears to be broken (for example, failing for reasons unrelated +to the change under test), please open an issue in the +[instructlab/instructlab](https://github.com/instructlab/instructlab/issues) +repository and include a link to the failing run. From 7edaff848a4053cd912690e3eb4d90770747dacd Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 13 Jul 2023 18:36:19 +0000 Subject: [PATCH 023/155] docs: tighten containerization intro wording --- docs/containerization.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/docs/containerization.md b/docs/containerization.md index 9b17543a70..4247821665 100644 --- a/docs/containerization.md +++ b/docs/containerization.md @@ -8,13 +8,12 @@ experience. ## Steps to build an image then run a container -The [`Containerfile`](../containers/cuda/Containerfile) -is based on Nvidia CUDA image, which lucky for us plugs -directly into Podman via their `nvidia-container-toolkit`! The `ubi9` base -image does not have most packages installed. The bulk of the `Containerfile` is -spent configuring your system so `ilab` can be installed and run properly. -`ubi9` as compared to `ubuntu` cannot install the entire `nvidia-12-4` toolkit. -This did not impact performance during testing. +The [`Containerfile`](../containers/cuda/Containerfile) is based on the Nvidia CUDA +image, which plugs directly into Podman via the `nvidia-container-toolkit`. The +`ubi9` base image does not have most packages installed, so the bulk of the +`Containerfile` is spent configuring your system so `ilab` can be installed and +run properly. Unlike `ubuntu`, `ubi9` cannot install the entire `nvidia-12-4` +toolkit. This did not impact performance during testing. ```shell 1. podman build -f From c391976d15a982ae7160d98c1478cd9b831c0964 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 20 Jul 2023 14:26:52 +0000 Subject: [PATCH 024/155] Add performance section to bug report template --- .github/ISSUE_TEMPLATE/bug_report.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index e912e092cf..118c77b168 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -39,6 +39,11 @@ file rather than pasting inline. --> +**Performance (if applicable)** + + **Additional context** From 2f6faf9c90a02b5330e5ccf0713a15ceaf62c74c Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 21 Jul 2023 14:39:38 +0000 Subject: [PATCH 025/155] Fix typo: 8GM -> 8GB in macOS troubleshooting note --- TROUBLESHOOTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md index 880be80cb6..39b0f603e4 100644 --- a/TROUBLESHOOTING.md +++ b/TROUBLESHOOTING.md @@ -6,7 +6,7 @@ This document is for commonly found problems and their solutions when using `ila ### `ilab generate` command running slow on macOS -If you notice `ilab generate` running for several hours or more on a Mac M-series, you should first check out the available memory on your system (See [Activity Monitor](https://support.apple.com/en-ie/guide/activity-monitor/welcome/mac) for more details). If there is < 8GM RAM available before serving a model, then check to see if you can free up some memory. +If you notice `ilab generate` running for several hours or more on a Mac M-series, you should first check out the available memory on your system (See [Activity Monitor](https://support.apple.com/en-ie/guide/activity-monitor/welcome/mac) for more details). If there is less than 8GB of RAM available before serving a model, then check to see if you can free up some memory. If this has not improved the running of the generation then check out [this discussion](https://github.com/ggerganov/llama.cpp/discussions/2182#discussioncomment-7698315). The suggestion here is to tweak the GPU limit of the macOS. By default it's around 60%-70% of your total RAM available, which is expressed as 0: From 9c942038f9f72e83eea35ec86f5ce3451e491503 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 24 Jul 2023 16:28:00 +0000 Subject: [PATCH 026/155] Clarify OS field label in bug report template --- .github/ISSUE_TEMPLATE/bug_report.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 118c77b168..0a139bcb28 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -30,7 +30,7 @@ file rather than pasting inline. --> **Device Info (please complete the following information):** - Hardware Specs: [e.g. Apple M2 Pro Chip, 16 GB Memory, etc.] - - OS Version: [e.g. Mac OS 14.4.1, Fedora Linux 40] + - OS and Version: [e.g. macOS 14.4.1, Fedora Linux 40] - Python Version: [output of `python --version`] - InstructLab Version: [output of `ilab --version`] - Accelerator / GPU (if applicable): [e.g. NVIDIA RTX 4090, AMD MI300X, none] From b6a5829f28343b13c182e93b78b215398cf2c897 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 28 Jul 2023 01:24:52 +0000 Subject: [PATCH 027/155] Mention related issues in feature request template --- .github/ISSUE_TEMPLATE/feature_request.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 6315abc659..20e94476f3 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -17,4 +17,4 @@ assignees: '' **Additional context** - + From d67c97983009285f970371f6062e7f8601fb1819 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 28 Jul 2023 17:22:40 +0000 Subject: [PATCH 028/155] docs: fix grammar in training next-step sentence --- notebooks/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/notebooks/README.md b/notebooks/README.md index 310b9f02e8..465111b728 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -36,7 +36,7 @@ Pre-requisites: 1. Setting up and training a LoRA. A LoRA uses Parameter Efficient Fine-tuning (PEFT) methods to fine-tune a model on a small subset of the overall parameters, allowing you to conduct fine-tuning in a fraction of the time, on a fraction of the hardware required. The resultant model should be updated and better handle your queries than the base model. 1. Inspecting the output model to make sure the LoRA training had the desired effect (i.e. the output has improved). -Once you have finished training and the output looks good, we encourage you go to stage, [Testing the fine-tuned model](../README.md#-test-the-newly-trained-model) +Once you have finished training and the output looks good, we encourage you to proceed to the next stage, [Testing the fine-tuned model](../README.md#-test-the-newly-trained-model). ### Kaggle (Unsupported and deprecated) From c1cca7f753cbe6535d7ab990fb84b600ed7d8920 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 2 Aug 2023 15:13:00 +0000 Subject: [PATCH 029/155] Add accessibility considerations section to feature request template --- .github/ISSUE_TEMPLATE/feature_request.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 20e94476f3..fae18d7a24 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -16,5 +16,8 @@ assignees: '' **Describe alternatives you've considered** +**Accessibility considerations** + + **Additional context** From 6af232f8d35124f63bcbb9a4f498b133cb4e354b Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 10 Aug 2023 02:21:07 +0000 Subject: [PATCH 030/155] docs(STABLE): clarify tag fetch and add CI check before promoting --- docs/STABLE.md | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/STABLE.md b/docs/STABLE.md index 3a295e8a74..3ae88c4bf4 100644 --- a/docs/STABLE.md +++ b/docs/STABLE.md @@ -7,19 +7,21 @@ To support this, we move the stable tag to the latest release, as needed. This can be done via the following steps: -Assume that the upstream repository (`instructlab/instructlab`) is using the `upstream` Git remote +Assume that the upstream repository (`instructlab/instructlab`) is using the `upstream` Git remote. Get current tags from the upstream: ```ShellSession $ git switch main -$ git fetch upstream -$ git pull upstream +$ git fetch upstream --tags +$ git pull upstream main # Just making sure you are up-to-date locally $ git tag --list $ git show-ref --tags ``` +Note: `git fetch upstream --tags` is important — without `--tags`, your local view of the `stable` tag may be stale, and you could end up force-pushing over a tag that someone else already moved. + Using v0.x.y as the example desired stable version: ```ShellSession @@ -27,10 +29,11 @@ $ git show v0.x.y ``` 1. Verify that this is the tag/commit you want for stable. -2. Double check with the short SHA hashes on tags [here](https://github.com/instructlab/instructlab/tags) -3. Copy the SHA hash +2. Double check with the short SHA hashes on tags [here](https://github.com/instructlab/instructlab/tags). +3. Confirm that CI passed on the release commit before promoting it to stable. +4. Copy the SHA hash. -I usually test first w/o --force and expect an error if I have everything right. +Test first w/o `--force` and expect an error if you have everything right: ```ShellSession $ git tag stable v0.x.y @@ -49,7 +52,7 @@ Verify the tag SHA hashes look correct: $ git show-ref --tags ``` -You can then push the new tag upstream with `-f` (force) flag - I usually test first without `--force` and expect an error if I have everything right. +You can then push the new tag upstream with the `-f` (force) flag. Again, test first without `--force` and expect an error if everything is right: ```ShellSession $ git push upstream stable @@ -61,7 +64,7 @@ hint: Updates were rejected because the tag already exists in the remote. If you are sure, push with force to upstream and prepare to live with the consequences of your actions. ```ShellSession -git push -f upstream stable +$ git push -f upstream stable ``` Finally, check the tags on the GitHub web UI to ensure everything looks correct. From b8eec347909a843bd1315545e7bc2e53e9184222 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 14 Aug 2023 14:34:22 +0000 Subject: [PATCH 031/155] docs: note GPUs used during containerization perf testing --- docs/containerization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/containerization.md b/docs/containerization.md index 4247821665..9594290dbc 100644 --- a/docs/containerization.md +++ b/docs/containerization.md @@ -13,7 +13,7 @@ image, which plugs directly into Podman via the `nvidia-container-toolkit`. The `ubi9` base image does not have most packages installed, so the bulk of the `Containerfile` is spent configuring your system so `ilab` can be installed and run properly. Unlike `ubuntu`, `ubi9` cannot install the entire `nvidia-12-4` -toolkit. This did not impact performance during testing. +toolkit. This did not impact performance during testing on Nvidia A100 and H100 GPUs. ```shell 1. podman build -f From 9fbe670797381b969d952fb85955d2c19fbb524f Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 16 Aug 2023 19:39:14 +0000 Subject: [PATCH 032/155] docs: clarify upstream source link in mlx_explore README --- src/instructlab/mlx_explore/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/instructlab/mlx_explore/README.md b/src/instructlab/mlx_explore/README.md index df9304a511..30772b99de 100644 --- a/src/instructlab/mlx_explore/README.md +++ b/src/instructlab/mlx_explore/README.md @@ -1,8 +1,8 @@ # Notice -The code in this folder is modified from the mlx-examples repository on GitHub. -For the original code, see [mlx-examples/llms](https://github.com/ml-explore/mlx-examples/tree/main/llms/gguf_llm) -with the following license. +The code in this folder is modified from the [mlx-examples](https://github.com/ml-explore/mlx-examples) repository on GitHub. +For the original code, see [mlx-examples/llms/gguf_llm](https://github.com/ml-explore/mlx-examples/tree/main/llms/gguf_llm), +which is distributed under the following license. ```license MIT License From 56bd79a0eef8eb99b606e195ba49bc240d1b1a67 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 18 Aug 2023 03:08:35 +0000 Subject: [PATCH 033/155] docs: add backend selection table and troubleshooting tips --- docs/gpu-acceleration.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/gpu-acceleration.md b/docs/gpu-acceleration.md index 4e98934e1e..55f1b2e24e 100644 --- a/docs/gpu-acceleration.md +++ b/docs/gpu-acceleration.md @@ -11,6 +11,19 @@ and `llama-cpp-python`. In short, you'll need to replace the default versions of these packages with versions that have been compiled for GPU-specific support, recompile `ilab`, then run it. +## Quick reference: backend selection + +Use the table below to pick the correct `llama-cpp-python` build flag for your +hardware. The corresponding `pip install` command in each section uses this +flag. + +| Hardware | Backend flag | Notes | +| ----------------------- | ------------------------ | -------------------------------------- | +| Nvidia GPU | `-DLLAMA_CUBLAS=on` | Requires CUDA toolkit 12.x | +| AMD GPU (ROCm) | `-DLLAMA_HIPBLAS=on` | Requires ROCm 5.7+ and hipBLAS | +| Apple Silicon | `-DLLAMA_METAL=on` | Default on macOS; usually preinstalled | +| Generic OpenCL | `-DLLAMA_CLBLAST=on` | Fallback for unsupported GPUs | + ## Python 3.11 (Linux only) > **NOTE:** This section may be outdated. At least AMD ROCm works fine with @@ -308,6 +321,21 @@ ggml_init_cublas: found 1 ROCm devices: Device 0: AMD Radeon RX 7900 XT, compute capability 11.0, VMM: no ``` +### Troubleshooting + +If inference speed seems unchanged after installing a GPU-enabled build, try the +following before opening an issue: + +- Confirm the rebuild actually picked up your backend flag by reinstalling with + `pip install -v ...` and watching the CMake output for the expected + `LLAMA_CUBLAS`/`LLAMA_HIPBLAS`/`LLAMA_METAL` line. +- Verify the wheel cache was cleared (`pip cache remove llama_cpp_python`); a + stale cached wheel is the most common cause of "nothing changed". +- Make sure no other Python environment is shadowing your venv by running + `which python` and `python -c 'import llama_cpp, sys; print(sys.executable, llama_cpp.__file__)'`. +- On ROCm, double-check that `HIP_VISIBLE_DEVICES` points at a discrete GPU and + not an integrated one. + ## Training `ilab train` also experimentally supports GPU acceleration on Linux. Details From daffbb746479a0f8343c08222b09b90ccc078c21 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sat, 19 Aug 2023 14:42:24 +0000 Subject: [PATCH 034/155] docs(habana): minor wording fixes in PyTorch stack section --- docs/habana-gaudi.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/habana-gaudi.md b/docs/habana-gaudi.md index 2d22eb005d..5c8fe8658c 100644 --- a/docs/habana-gaudi.md +++ b/docs/habana-gaudi.md @@ -108,7 +108,7 @@ chmod +x habanalabs-installer.sh > **NOTE** > -> Habana Labs Installer 1.15.1 only supports RHEL 9.2 and will fail on 9.3+. You can hack around the limitation by patching the installer: +> Habana Labs Installer 1.15.1 only supports RHEL 9.2 and will fail on 9.3+. You can work around the limitation by patching the installer: > > ```shell > sed -i 's/OS_VERSION=\$VERSION_ID/OS_VERSION=9.2/' habanalabs-installer.sh @@ -136,9 +136,9 @@ Validate installation: ## Habana Lab's PyTorch stack -Habana Labs comes with a modified fork of PyTorch that is build with Intel's oneAPI Math Kernel Library (oneMKL). The actual HPU bindings and helpers are provided by the the `habana_framework` package. Imports of `habana_framework` sub-packages register `hpu` device support, `torch.hpu` module, and `dynamo` backends. +Habana Labs comes with a modified fork of PyTorch that is built with Intel's oneAPI Math Kernel Library (oneMKL). The actual HPU bindings and helpers are provided by the `habana_framework` package. Imports of `habana_framework` sub-packages register `hpu` device support, the `torch.hpu` module, and `dynamo` backends. -The [`SFTTrainer`](https://huggingface.co/docs/trl/sft_trainer) from `trl` does not work with Habana stack. Instead the `GaudiSFTTrainer` from [optimum-habana](https://huggingface.co/docs/optimum/habana/index) is needed. The version on PyPI is currently broken, but the HabanaAI [optimum-habana-fork](https://github.com/HabanaAI/optimum-habana-fork) works. +The [`SFTTrainer`](https://huggingface.co/docs/trl/sft_trainer) from `trl` does not work with the Habana stack. Instead the `GaudiSFTTrainer` from [optimum-habana](https://huggingface.co/docs/optimum/habana/index) is needed. The version on PyPI is currently broken, but the HabanaAI [optimum-habana-fork](https://github.com/HabanaAI/optimum-habana-fork) works. ## Install and run InstructLab with Intel Gaudi From a21be8c510c8b4d4ee1a1239063805cab0d4a3c4 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 6 Sep 2023 15:06:17 +0000 Subject: [PATCH 035/155] docs: expand MAINTAINERS with contact and governance links --- MAINTAINERS.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 5d436787ed..a2482e96c7 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -1,3 +1,15 @@ # InstructLab Maintainers For a complete list of InstructLab project Maintainers, see the [Maintainers list](https://github.com/instructlab/community/blob/main/MAINTAINERS.md) in the `instructlab/community` repository. + +## Reporting Issues + +If you need to contact the maintainers of this repository, please: + +- Open an [issue](https://github.com/instructlab/sdg/issues) for bugs or feature requests. +- Start a [discussion](https://github.com/instructlab/sdg/discussions) for general questions. +- See [SECURITY.md](./SECURITY.md) for reporting security vulnerabilities. + +## Governance + +Project governance, including the process for becoming a maintainer, is documented in the [instructlab/community](https://github.com/instructlab/community) repository. From 49881fe860ba927fe129c83bf8baf13bbfb17391 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 14 Sep 2023 13:05:10 +0000 Subject: [PATCH 036/155] Fix typo: Rogue -> Rouge in troubleshooting docs --- TROUBLESHOOTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md index 39b0f603e4..1c9721ad4f 100644 --- a/TROUBLESHOOTING.md +++ b/TROUBLESHOOTING.md @@ -65,7 +65,7 @@ The data generation step is executed via the `ilab generate` command, and is res 1. Increase the number of instructions generated by passing the `--num-instructions` flag to the `ilab generate` command as follows: `ilab generate --num-instructions 1000`. The `--num-instructions` flag will generate 1000 points of synthetic data based on your provided examples. The greater the number of instructions generated, the better the model will be trained (within reasonable limits). -2. Adjust the rouge threshold via `--rouge-threshold` parameter. Rogue threshold is a parameter that determines how likely a synthetic data point, generated by the model, will be added to the output based on how similar it is to previously generated data points. +2. Adjust the rouge threshold via `--rouge-threshold` parameter. Rouge threshold is a parameter that determines how likely a synthetic data point, generated by the model, will be added to the output based on how similar it is to previously generated data points. The value of rouge threshold ranges from 1.0 to 0.0, where 1.0 indicates maximum leniency (every newly generated data point is accepted) and 0.0 indicates maximum strictness (every newly generated data point is rejected). Setting a rouge threshold value closer to 0.0 would force the model to generate data that is different from what it has already generated, leading to a more diverse dataset overall. Rouge threshold can be set as follows: `ilab generate --rouge-threshold 0.75` 3. Using a better model via `--model`. Larger models can lead to better data generation. This option requires users to be familiar with various existing models, and which specific models would suit their needs. This could mean either using a model with more nodes than the default InstructLab `merlinite-7b-lab` model, such as the `Mixtral-8x7B-Instruct-v0.1` model, or using an unquantized version of the InstructLab `merlinite-7b-lab` model. It can be used as follows: `ilab generate --model Mixtral-8x7B-Instruct-v0.1` From 9740f16692e684282282acb60f992e3f8af8d531 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 14 Sep 2023 16:03:19 +0000 Subject: [PATCH 037/155] Expand performance section of bug report template --- .github/ISSUE_TEMPLATE/bug_report.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 0a139bcb28..a37d617218 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -40,9 +40,16 @@ file rather than pasting inline. --> secrets removed) and note any non-default settings. --> **Performance (if applicable)** - + **Additional context** - OS and Version: [e.g. macOS 14.4.1, Fedora Linux 40] - Python Version: [output of `python --version`] - InstructLab Version: [output of `ilab --version`] + - Installation method: [e.g. `pip install`, from source, container] - Accelerator / GPU (if applicable): [e.g. NVIDIA RTX 4090, AMD MI300X, none] **Configuration** From 1e3feb451f7748bf72ee63e1d517c6e60d7a0b64 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 16 Oct 2023 13:26:40 +0000 Subject: [PATCH 045/155] docs(notebooks): clarify Colab vs Kaggle guidance and tidy steps --- notebooks/README.md | 39 +++++++++++++++++++-------------------- 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/notebooks/README.md b/notebooks/README.md index 465111b728..8e6e8e1d1d 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -11,15 +11,15 @@ Also, there is active work being done to support Linux, so if you have access to a Linux machine with GPUs, that might also be a better option. Next, you'll get to fine-tune a LoRA (Low-Rank Adaptation of Large Language -Models) using a Jupyter notebook and (preferably) Google Colab platform or if -unable to use Colab, unmaintained instructions for -[Kaggle](https://www.kaggle.com). +Models) using a Jupyter notebook and (preferably) the Google Colab platform. +If you are unable to use Colab, unmaintained instructions for +[Kaggle](https://www.kaggle.com) are also provided below. We've laid out the steps to get started with either platform below. ## Setting up the notebook -### Google Colab +### Google Colab (recommended) Pre-requisites: @@ -32,13 +32,15 @@ Pre-requisites: 1. [Open the notebook in Google Colab](https://colab.research.google.com/github/instructlab/instructlab/blob/main/notebooks/Training_a_LoRA_With_Instruct_Lab.ipynb) 1. Uploading the output of `ilab generate` (a synthetic dataset created based on your hand written prompts/responses). -1. Checking the base model before training +1. Checking the base model before training. 1. Setting up and training a LoRA. A LoRA uses Parameter Efficient Fine-tuning (PEFT) methods to fine-tune a model on a small subset of the overall parameters, allowing you to conduct fine-tuning in a fraction of the time, on a fraction of the hardware required. The resultant model should be updated and better handle your queries than the base model. 1. Inspecting the output model to make sure the LoRA training had the desired effect (i.e. the output has improved). Once you have finished training and the output looks good, we encourage you to proceed to the next stage, [Testing the fine-tuned model](../README.md#-test-the-newly-trained-model). -### Kaggle (Unsupported and deprecated) +### Kaggle (unsupported and deprecated) + +> **Note:** The Kaggle workflow is no longer actively maintained. Prefer Google Colab or local `ilab train` where possible. Using a [Kaggle Notebook](https://github.com/instructlab/instructlab/blob/main/notebooks/Training_a_LoRA_With_Instruct_Lab.ipynb) and the NVIDIA P100 provided in the free tier, we will fine tune a LoRA. @@ -48,37 +50,34 @@ and the NVIDIA P100 provided in the free tier, we will fine tune a LoRA. 1. You'll need a Kaggle account, which you can create by visiting [Kaggle's Sign-up Page](https://www.kaggle.com/account/login?phase=startRegisterTab&returnUrl=%2F). 1. To use Kaggle's accelerators, you'll have to verify your account with a phone number. Visit the [account settings page](https://www.kaggle.com/settings) and select "Phone Verification". -[!NOTE] -At present, you'll need to download the notebook and upload it to Kaggle. The following steps will walk you through uploading the notebook. Once this repository is open sourced, we will make an 'Open in Colab' button** - #### Uploading the notebook Once you have Kaggle properly configured, you can then run this notebook by following this process: -1. At the top-left of the Kaggle page, click the "Create" button +1. At the top-left of the Kaggle page, click the "Create" button. -![create-notebook](images/kaggle/create.png) + ![create-notebook](images/kaggle/create.png) -2. Then, select "Notebook" from the Dropdown menu. +2. Then, select "Notebook" from the dropdown menu. -![create-new-notebook](images/kaggle/create-new-nb.png) + ![create-new-notebook](images/kaggle/create-new-nb.png) 3. This will create a new notebook with some example data inside of it already. From here, select "File" at the top left corner. -![new-notebook-file-click](images/kaggle/file-click.png) + ![new-notebook-file-click](images/kaggle/file-click.png) 4. Then, select "Import notebook". This will prompt you to upload a file from a local disk (you can also use GitHub). -![import-new-notebook](images/kaggle/import-nb.png) + ![import-new-notebook](images/kaggle/import-nb.png) -5. With the notebook uploaded, we'll then need to click the three vertical dots on the top right to open the accelerator options. +5. With the notebook uploaded, click the three vertical dots on the top right to open the accelerator options. -![select-an-accelerator](images/kaggle/select-accelerator.png) + ![select-an-accelerator](images/kaggle/select-accelerator.png) 6. Then select the **P100 GPU Accelerator**. The other accelerator options will not work (yet). -![selecting-the-p100-gpu](images/kaggle/select-accelerator-p100.png) + ![selecting-the-p100-gpu](images/kaggle/select-accelerator-p100.png) -7. Finally, make sure to click "Restart & Clear Cell Outputs" before you run. **_KAGGLE WILL NOT LET YOU RUN NOTEBOOKS OVER 1 MEGABYTE IN SIZE_** +7. Finally, make sure to click "Restart & Clear Cell Outputs" before you run. **_KAGGLE WILL NOT LET YOU RUN NOTEBOOKS OVER 1 MEGABYTE IN SIZE._** -![restart-and-clear-cell-outputs](images/kaggle/clear-outputs.png) + ![restart-and-clear-cell-outputs](images/kaggle/clear-outputs.png) From 147c97e8cb373714738e461cbd92284731118363 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 24 Oct 2023 21:00:16 +0000 Subject: [PATCH 046/155] Document project-specific roles and decision making in governance --- governance.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/governance.md b/governance.md index a52fec09d2..7f2a7a5787 100644 --- a/governance.md +++ b/governance.md @@ -1,3 +1,25 @@ # InstructLab Governance For information about how the InstructLab project governance operates, see the [InstructLab Governance](https://github.com/instructlab/community/blob/main/governance.md#instructlab-governance) document in the `instructlab/community` repository. + +## Project-Specific Roles + +While overall governance is defined in the `instructlab/community` repository, this repository recognizes the following project-specific roles: + +- **Maintainers**: Responsible for reviewing and merging pull requests, triaging issues, and guiding the technical direction of this repository. See [MAINTAINERS.md](MAINTAINERS.md) if present, or the `CODEOWNERS` file for the current list. +- **Reviewers**: Trusted contributors who regularly review pull requests and provide feedback. Reviewers may be promoted to maintainers based on sustained contributions. +- **Contributors**: Anyone who contributes code, documentation, tests, or other improvements. See [CONTRIBUTING.md](CONTRIBUTING.md) for how to get started. + +## Decision Making + +Day-to-day decisions (bug fixes, small features, documentation updates) are made by maintainers via the standard pull request review process. Larger or more impactful changes — such as architectural shifts, breaking API changes, or new subprojects — should be discussed in a GitHub issue or in the community meeting before implementation begins, to give the broader community an opportunity to weigh in. + +When consensus cannot be reached among maintainers, the matter is escalated to the InstructLab Oversight Committee as described in the [community governance document](https://github.com/instructlab/community/blob/main/governance.md#instructlab-governance). + +## Reporting Concerns + +Governance-related concerns, including questions about maintainer conduct or project direction, can be raised by: + +1. Opening an issue in this repository (for technical or process matters), or +2. Contacting the InstructLab Oversight Committee directly (for sensitive matters), or +3. Following the process described in the project [Code of Conduct](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md). From 99f9e721f14be5fd9ceba1c1d6879b910127db3f Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 27 Oct 2023 19:44:02 +0000 Subject: [PATCH 047/155] Mention focus management in a11y considerations --- .github/ISSUE_TEMPLATE/feature_request.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index fae18d7a24..ec9f804f67 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -17,7 +17,7 @@ assignees: '' **Accessibility considerations** - + **Additional context** From cf4b68021a52c8eb067c0a3e40fc8dda9265d001 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 31 Oct 2023 16:18:53 +0000 Subject: [PATCH 048/155] docs: add TOC and issue-reporting section to troubleshooting --- TROUBLESHOOTING.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md index 1c9721ad4f..616fe90bf6 100644 --- a/TROUBLESHOOTING.md +++ b/TROUBLESHOOTING.md @@ -2,6 +2,16 @@ This document is for commonly found problems and their solutions when using `ilab`. There is also a section that includes information on fine-tuning and troubleshooting your model to optimize the quality of its responses. +## Table of contents + +- [`ilab` troubleshooting](#ilab-troubleshooting) + - [`ilab generate` command running slow on macOS](#ilab-generate-command-running-slow-on-macos) +- [Model fine tuning and response optimization](#model-fine-tuning-and-response-optimization) + - [Skill composition](#skill-composition) + - [Data generation](#data-generation) + - [Training](#training) +- [Additional resources](#additional-resources) + ## `ilab` troubleshooting ### `ilab generate` command running slow on macOS @@ -89,3 +99,12 @@ The training step is run with the `ilab train` command. This step trains the mod - InstructLab Community [FAQ](https://github.com/instructlab/community/blob/main/FAQ.md) - InstructLab Taxonomy [FAQ](https://github.com/instructlab/taxonomy/discussions/538) - Discussion [board](https://github.com/instructlab/instructlab/discussions) + +## Reporting new issues + +If the steps above did not resolve your problem, please open an issue on the [InstructLab issue tracker](https://github.com/instructlab/instructlab/issues). When filing a new report, include the following so maintainers can help you faster: + +- The exact `ilab` command you ran and its full output. +- The output of `ilab system info` (or, if unavailable, your OS, architecture, Python version, and `ilab --version`). +- Hardware details relevant to the issue (CPU/GPU, total RAM, accelerator type). +- A minimal reproduction (e.g., a small skill YAML) when applicable. From 96128c8e409a75ad0a4929e7bfbc298591b8091c Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 8 Nov 2023 01:37:23 +0000 Subject: [PATCH 049/155] docs(coc): expand scope list and reporting checklist --- CODE_OF_CONDUCT.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 3c9defb7ce..ce07ad5e39 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -13,6 +13,7 @@ When reporting an issue, it is helpful to include: - The names or handles of any individuals involved, if known. - Links to any public records of the behavior (e.g., comments, commits, messages), if available. - Whether you believe the incident is ongoing or has been resolved. +- Any immediate safety concerns that maintainers should be aware of. Reports are handled confidentially by the InstructLab community maintainers. See the linked Code of Conduct for the current list of contacts and the full reporting process. @@ -20,7 +21,15 @@ If you are unsure whether something rises to the level of a Code of Conduct viol ## Scope -This Code of Conduct applies within all project spaces — including the code repository, issue tracker, pull requests, discussions, and any official communication channels — as well as when an individual is officially representing the project in public spaces. +This Code of Conduct applies within all project spaces — including, but not limited to: + +- the code repository and its issue tracker, +- pull requests and code review discussions, +- GitHub Discussions and any project mailing lists, +- official chat channels (e.g., Slack, Discord, Matrix), and +- project-related events, meetups, and conference talks. + +It also applies when an individual is officially representing the project in public spaces. ## Acknowledgements From da67590d2de43c898a2302376dace83b0ecb119c Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 13 Nov 2023 14:42:22 +0000 Subject: [PATCH 050/155] docs(rocm): minor wording fix in ilab train note --- containers/rocm/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/containers/rocm/README.md b/containers/rocm/README.md index d9c85d5bcc..c1e62bdc3b 100644 --- a/containers/rocm/README.md +++ b/containers/rocm/README.md @@ -25,8 +25,8 @@ inference will silently fall back to CPU, which is dramatically slower. To update InstructLab CLI to latest version: `pip install -e ~/path/to/instructlab/instructlab` `ilab generate` and `ilab chat` use the GPU automatically. `ilab train` needs -more powerful and recent GPU and therefore does not use GPU by default. To -train on a GPU, run `ilab train --device cuda`. +a more powerful and recent GPU and therefore does not use the GPU by default. +To train on a GPU, run `ilab train --device cuda`. You can verify that the GPU is visible inside the toolbox by running `rocminfo | grep -E 'Name|gfx'` and confirming that your card's `gfx` target From d625ba19eaab822c19787c3ea0f293197bde87cf Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 16 Nov 2023 16:39:21 +0000 Subject: [PATCH 051/155] Add reporting guidance and amendments section to governance --- governance.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/governance.md b/governance.md index 7f2a7a5787..86a0f47bf9 100644 --- a/governance.md +++ b/governance.md @@ -23,3 +23,9 @@ Governance-related concerns, including questions about maintainer conduct or pro 1. Opening an issue in this repository (for technical or process matters), or 2. Contacting the InstructLab Oversight Committee directly (for sensitive matters), or 3. Following the process described in the project [Code of Conduct](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md). + +When reporting, please include enough context (links to issues, pull requests, or discussions) to allow the recipients to understand the situation, but avoid sharing private information about other contributors without their consent. + +## Amendments + +This document may be updated over time to reflect changes in project practice. Substantive changes should be proposed via pull request and require approval from a majority of active maintainers before being merged. Minor edits (typos, formatting, broken links) may be merged under the normal review process. From b97adc1cd635e3c3203040382d0112144cd5b00c Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 21 Nov 2023 01:04:29 +0000 Subject: [PATCH 052/155] docs: clarify T4 VRAM wording in Colab pre-reqs --- notebooks/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/notebooks/README.md b/notebooks/README.md index 8e6e8e1d1d..9fd2d61a22 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -24,7 +24,7 @@ We've laid out the steps to get started with either platform below. Pre-requisites: * [Google Colab](https://research.google.com/colaboratory/faq.html) -* A Gmail account that you're logged into. This will allow you to use Google Colab, which in the free tier will give you access to an NVidia T4 x 15GB GPU. +* A Gmail account that you're logged into. This will allow you to use Google Colab, which in the free tier will give you access to an NVidia T4 with 15GB of VRAM. ## Running the notebook From aba973978077dc69d7eb71a96cad4240049851e9 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 22 Nov 2023 14:26:40 +0000 Subject: [PATCH 053/155] docs: note upstream history is preserved in git blame --- src/instructlab/train/lora_mlx/models/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/instructlab/train/lora_mlx/models/README.md b/src/instructlab/train/lora_mlx/models/README.md index 9e9507c5a5..1784345c67 100644 --- a/src/instructlab/train/lora_mlx/models/README.md +++ b/src/instructlab/train/lora_mlx/models/README.md @@ -1,6 +1,6 @@ # Notice -The code in this folder is modified from the [mlx-examples lora/models](https://github.com/ml-explore/mlx-examples/tree/main/lora/models) directory, distributed under the MIT License reproduced below. +The code in this folder is modified from the [mlx-examples lora/models](https://github.com/ml-explore/mlx-examples/tree/main/lora/models) directory (upstream commit history preserved in git blame), distributed under the MIT License reproduced below. ```license MIT License From 5b6cf37f7582bfe84b5bef725f1a2c43eccb0ecb Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sun, 3 Dec 2023 19:17:15 +0000 Subject: [PATCH 054/155] docs: use descriptive link text in generator README notice --- src/instructlab/generator/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/instructlab/generator/README.md b/src/instructlab/generator/README.md index db12d09eba..967dfc3ef3 100644 --- a/src/instructlab/generator/README.md +++ b/src/instructlab/generator/README.md @@ -1,6 +1,6 @@ ### Notice -This folder uses code from the `stanford_alpaca` project developed by Tatsu Lab, available [here](https://github.com/tatsu-lab/stanford_alpaca). The original code is licensed under the Apache License: +This folder uses code from the [`stanford_alpaca`](https://github.com/tatsu-lab/stanford_alpaca) project developed by Tatsu Lab. The original code is licensed under the Apache License, Version 2.0: ```license Apache License @@ -204,4 +204,4 @@ This folder uses code from the `stanford_alpaca` project developed by Tatsu Lab, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -``` \ No newline at end of file +``` From 4fdc3b91cb2ae451e7e7bf317b96c9cc1f2dbb06 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sat, 9 Dec 2023 02:13:12 +0000 Subject: [PATCH 055/155] Link CODEOWNERS file directly in governance roles section --- governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/governance.md b/governance.md index 86a0f47bf9..2fc079eb07 100644 --- a/governance.md +++ b/governance.md @@ -6,7 +6,7 @@ For information about how the InstructLab project governance operates, see the [ While overall governance is defined in the `instructlab/community` repository, this repository recognizes the following project-specific roles: -- **Maintainers**: Responsible for reviewing and merging pull requests, triaging issues, and guiding the technical direction of this repository. See [MAINTAINERS.md](MAINTAINERS.md) if present, or the `CODEOWNERS` file for the current list. +- **Maintainers**: Responsible for reviewing and merging pull requests, triaging issues, and guiding the technical direction of this repository. See [MAINTAINERS.md](MAINTAINERS.md) if present, or the [CODEOWNERS](CODEOWNERS) file for the current list. - **Reviewers**: Trusted contributors who regularly review pull requests and provide feedback. Reviewers may be promoted to maintainers based on sustained contributions. - **Contributors**: Anyone who contributes code, documentation, tests, or other improvements. See [CONTRIBUTING.md](CONTRIBUTING.md) for how to get started. From dd72582b8c78e2b32bd58926682b601233d5d7f6 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 11 Dec 2023 15:14:45 +0000 Subject: [PATCH 056/155] docs: update SECURITY.md link to community repo --- SECURITY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/SECURITY.md b/SECURITY.md index 5c45c64706..ca6035755e 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,3 +1,3 @@ # Security Policy -You can find information on how to report a potential security vulnerability, as well as where to subscribe to receive security alerts, on the InstructLab project's [Security Page](https://github.com/instructlab/.github/blob/main/SECURITY.md). +You can find information on how to report a potential security vulnerability, as well as where to subscribe to receive security alerts, on the InstructLab project's [Security Page](https://github.com/instructlab/community/blob/main/SECURITY.md). From 8d370045cf04b7bdc58f24b52914fc1bf3bc4c3e Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 11 Dec 2023 21:52:21 +0000 Subject: [PATCH 057/155] docs(coc): advise against public disclosure before maintainer reply --- CODE_OF_CONDUCT.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index ce07ad5e39..587cbea9e7 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -15,7 +15,7 @@ When reporting an issue, it is helpful to include: - Whether you believe the incident is ongoing or has been resolved. - Any immediate safety concerns that maintainers should be aware of. -Reports are handled confidentially by the InstructLab community maintainers. See the linked Code of Conduct for the current list of contacts and the full reporting process. +Reports are handled confidentially by the InstructLab community maintainers. See the linked Code of Conduct for the current list of contacts and the full reporting process. Please avoid posting details of a suspected violation in public issues or pull requests before maintainers have had a chance to respond privately. If you are unsure whether something rises to the level of a Code of Conduct violation, you are still encouraged to reach out — maintainers would rather hear about a concern early than learn about it after it has escalated. From 96660b64386b89c57de31117ee60a009bd6434ff Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 18 Dec 2023 23:05:12 +0000 Subject: [PATCH 058/155] Add regression and disk-space fields to bug report template --- .github/ISSUE_TEMPLATE/bug_report.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index b30119269a..1d8b5a5a24 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -35,11 +35,17 @@ file rather than pasting inline. --> - InstructLab Version: [output of `ilab --version`] - Installation method: [e.g. `pip install`, from source, container] - Accelerator / GPU (if applicable): [e.g. NVIDIA RTX 4090, AMD MI300X, none] + - Available disk space: [e.g. `df -h` for the working directory] **Configuration** +**Regression** + + **Performance (if applicable)** **Which issue is resolved by this Pull Request:** Resolves # -**Description of your changes:** +## Description of Changes + + + +## Type of Change + + + +- [ ] Bug fix (non-breaking change which fixes an issue) +- [ ] New feature (non-breaking change which adds functionality) +- [ ] Breaking change (fix or feature that would cause existing functionality to change) +- [ ] Documentation update +- [ ] Refactor / code cleanup +- [ ] Build, CI, or tooling change -**Checklist:** +## Testing + + + +- [ ] Existing tests pass locally +- [ ] New tests have been added to cover the changes +- [ ] Manual verification was performed (describe below) + +## Checklist + +- [ ] I have read the contributing guidelines +- [ ] My code follows the project's style guidelines +- [ ] I have performed a self-review of my own code +- [ ] I have commented my code in hard-to-understand areas - [ ] Tests added or updated - [ ] Documentation updated (if applicable) +- [ ] No new warnings or errors have been introduced + +## Screenshots / Additional Notes + + From 7daa3cf4cb446cdc2eb34ccf5e517fd1e58addaa Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 8 Jan 2024 16:17:39 +0000 Subject: [PATCH 067/155] docs: fix dangling 'or' in reporting concerns list --- governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/governance.md b/governance.md index 8b44f835cb..8f5175ccec 100644 --- a/governance.md +++ b/governance.md @@ -23,7 +23,7 @@ When consensus cannot be reached among maintainers, the matter is escalated to t Governance-related concerns, including questions about maintainer conduct or project direction, can be raised by: -1. Opening an issue in this repository (for technical or process matters), or +1. Opening an issue in this repository (for technical or process matters), 2. Contacting the InstructLab Oversight Committee directly (for sensitive matters), or 3. Following the process described in the project [Code of Conduct](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md). From 5ec9d4a8f77e321a51c25989cdebb6a0f07e3274 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 10 Jan 2024 02:45:34 +0000 Subject: [PATCH 068/155] Expand performance section of bug report template --- .github/ISSUE_TEMPLATE/bug_report.md | 37 ++++++++++++++++++++++------ 1 file changed, 30 insertions(+), 7 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 1d8b5a5a24..0e35d84cb7 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -48,15 +48,38 @@ the bug. --> **Performance (if applicable)** + `rocm-smi`, `top`, or `htop` output). Sustained vs. bursty usage + is helpful to note. + - Disk I/O or network activity if the workload is I/O-bound. + + Workload shape: + - Dataset / model size and any batch size, sequence length, or + concurrency settings. + - Number of GPUs / workers and any relevant distributed settings. + - Quantization, precision (fp16/bf16/fp32), or other tuning flags. + + Reproducibility: + - Whether the slowdown is reproducible on a fresh environment. + - Whether it occurs consistently or only intermittently, and how + many runs you have observed. + - A minimal command or script that reproduces the regression, if + you have one. --> **Additional context** +**Actual behavior** + + **Screenshots or Logs** - InstructLab Version: [output of `ilab --version`] - Installation method: [e.g. `pip install`, from source, container] - Accelerator / GPU (if applicable): [e.g. NVIDIA RTX 4090, AMD MI300X, none] + - Accelerator driver / runtime version (if applicable): [e.g. CUDA 12.4, + ROCm 6.1, Metal] - Available disk space: [e.g. `df -h` for the working directory] **Configuration** From 4ce47dd5e72787675dd6d5f5ce5d872dfc1195c0 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 1 Jul 2024 15:52:36 +0000 Subject: [PATCH 082/155] Mention reduced motion and mockups in feature request template --- .github/ISSUE_TEMPLATE/feature_request.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index ec9f804f67..454bd87f01 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -17,7 +17,7 @@ assignees: '' **Accessibility considerations** - + **Additional context** - + From 93ae6ce0b79abf9800db3be5721489a7895783ab Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 5 Jul 2024 14:39:16 +0000 Subject: [PATCH 083/155] docs: clarify meaning of the `stable` tag in release strategy --- docs/release-strategy.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/release-strategy.md b/docs/release-strategy.md index 4dfdcd9340..66671c9165 100644 --- a/docs/release-strategy.md +++ b/docs/release-strategy.md @@ -35,6 +35,9 @@ Every `X.Y` release stream gets a new branch named `release-vX.Y`. Each release, `X.Y.Z`, exists as a tag named `vX.Y.Z`. +The `stable` tag always points at the most recently validated release and is +moved forward as part of the release mechanics described below. + ## Release Branch Maintenance Maintenance efforts are only on the most recent Y-stream. From 97fb9c4902865f04874b06c1a2c11ae3eef11f82 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 8 Jul 2024 16:46:02 +0000 Subject: [PATCH 084/155] docs(skill-wiki): add plateaus/regression and robustness measure --- tests/testdata/temp_repo/docs/skill-wiki.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/testdata/temp_repo/docs/skill-wiki.md b/tests/testdata/temp_repo/docs/skill-wiki.md index e5235948b1..4b106f6688 100644 --- a/tests/testdata/temp_repo/docs/skill-wiki.md +++ b/tests/testdata/temp_repo/docs/skill-wiki.md @@ -22,6 +22,10 @@ Hard skills are typically teachable and measurable (e.g., typing speed, fluency Skill acquisition is commonly described as progressing through several stages, often summarized as cognitive, associative, and autonomous. In the cognitive stage, the learner relies heavily on explicit instruction and conscious attention. In the associative stage, performance becomes more consistent as errors are reduced through feedback and repetition. In the autonomous stage, execution becomes largely automatic, freeing cognitive resources for higher-level tasks. Deliberate practice — focused, goal-directed effort with timely feedback — is widely regarded as a key driver of progression between these stages. +### Plateaus and regression + +Progress through these stages is rarely smooth. Learners commonly encounter *plateaus*, during which measurable improvement stalls despite continued practice. Plateaus can reflect the consolidation of a skill at a stable level, the need to restructure an underlying technique before further gains are possible, or simply diminishing returns from a particular training regime. Short-term *regression* — a temporary dip in performance — is also typical when a learner deliberately rebuilds a sub-skill or adopts a new strategy. + ## Measuring skill performance Assessing how skillfully a task is performed typically combines several complementary measures: @@ -31,6 +35,7 @@ Assessing how skillfully a task is performed typically combines several compleme - **Consistency**: variance of outcomes across repeated attempts under similar conditions. - **Efficiency**: the cognitive, physical, or material resources expended relative to the result. - **Transfer**: the degree to which the skill generalizes to novel but related situations. +- **Robustness**: the degree to which performance is preserved under stress, fatigue, distraction, or unfamiliar conditions. Because these dimensions can trade off against one another — for example, a practitioner may sacrifice speed for accuracy, or vice versa — meaningful evaluation usually reports several of them together rather than collapsing performance into a single score. @@ -41,7 +46,9 @@ Because these dimensions can trade off against one another — for example, a pr - Expertise - Learning - Practice (learning method) +- Skill assessment - Training +- Transfer of learning ## References @@ -49,3 +56,4 @@ Because these dimensions can trade off against one another — for example, a pr 2. U.S. Department of Labor, Secretary's Commission on Achieving Necessary Skills (SCANS) report. 3. Katz, R. L. (1955). "Skills of an Effective Administrator". *Harvard Business Review*. 4. Laker, D. R.; Powell, J. L. (2011). "The differences between hard and soft skills and their relative impact on training transfer". *Human Resource Development Quarterly*. +5. Ericsson, K. A.; Krampe, R. T.; Tesch-Römer, C. (1993). "The role of deliberate practice in the acquisition of expert performance". *Psychological Review*. From 94c1ef2be11d114d7718d07d74c4bf98332c4b15 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 12 Jul 2024 19:21:16 +0000 Subject: [PATCH 085/155] docs: include video calls in Code of Conduct scope --- CODE_OF_CONDUCT.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 6cfc5060a9..1d3a4028fb 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -27,7 +27,8 @@ This Code of Conduct applies within all project spaces — including, but not li - pull requests and code review discussions, - GitHub Discussions and any project mailing lists, - official chat channels (e.g., Slack, Discord, Matrix), -- project-related events, meetups, and conference talks, and +- project-related events, meetups, and conference talks, +- video calls and working-group meetings held under the project's banner, and - social media accounts operated on behalf of the project. It also applies when an individual is officially representing the project in public spaces. From dae3808ea949a696968b542c4fdb1f383756592a Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 23 Jul 2024 19:57:04 +0000 Subject: [PATCH 086/155] docs(governance): clarify redaction guidance when reporting concerns --- governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/governance.md b/governance.md index 8f5175ccec..bfaedc634e 100644 --- a/governance.md +++ b/governance.md @@ -27,7 +27,7 @@ Governance-related concerns, including questions about maintainer conduct or pro 2. Contacting the InstructLab Oversight Committee directly (for sensitive matters), or 3. Following the process described in the project [Code of Conduct](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md). -When reporting, please include enough context (links to issues, pull requests, or discussions) to allow the recipients to understand the situation, but avoid sharing private information about other contributors without their consent. +When reporting, please include enough context (links to issues, pull requests, or discussions) to allow the recipients to understand the situation, but avoid sharing private information about other contributors without their consent. If in doubt, redact identifying details and let the recipients ask follow-up questions privately. ## Amendments From 1657e551b107c79eccb545746b89a7f101a82e7f Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 24 Jul 2024 13:31:26 +0000 Subject: [PATCH 087/155] docs: add CLI export and troubleshooting sections --- docs/README.md | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/docs/README.md b/docs/README.md index 80d1906277..ccd21fc5b5 100644 --- a/docs/README.md +++ b/docs/README.md @@ -38,3 +38,36 @@ pull requests. After editing a `.puml` source, export the rendered PNG/SVG alongside the source file and commit both. This avoids forcing every reader to re-run the renderer locally. + +### Command-line export + +If you have the PlantUML JAR installed, you can regenerate figures from the +command line without opening VS Code: + +```bash +# Render a single file to PNG (default) +java -jar plantuml.jar workflow.puml + +# Render to SVG, which is preferred for diagrams embedded in web docs +java -jar plantuml.jar -tsvg workflow.puml + +# Render every .puml file in the current directory +java -jar plantuml.jar -tsvg "*.puml" +``` + +For ditaa-based figures specifically, pass `-tlatex` or `-tpng` as appropriate; +ditaa output is raster-only, so SVG is not available for those diagrams. + +## Troubleshooting + +- **Blank preview in VS Code.** Ensure Java is on your `PATH` and Graphviz + (`dot`) is installed for non-ditaa diagrams. The plugin's output panel will + usually report the missing dependency. +- **"Dot executable not found."** Install Graphviz (`brew install graphviz`, + `apt install graphviz`, or the Windows installer) and restart VS Code so the + updated `PATH` is picked up. +- **Diff noise in PRs.** PlantUML embeds a timestamp in PNG metadata by + default. Pass `-nometadata` when exporting to keep diffs minimal. +- **Fonts differ between machines.** Pin a font explicitly in the diagram + (e.g. `skinparam defaultFontName "DejaVu Sans"`) so renders are reproducible + across contributors. From ec7d3b89128c3d5e33e1122d786dc7cb54ada7d2 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sat, 27 Jul 2024 01:30:35 +0000 Subject: [PATCH 088/155] Add Risk Assessment section to PR template --- .github/PULL_REQUEST_TEMPLATE.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 37ab7acafe..df72e56faa 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -36,6 +36,13 @@ reviewers can reproduce. Include relevant details about your test setup. - [ ] New tests have been added to cover the changes - [ ] Manual verification was performed (describe below) +## Risk Assessment + + + ## Checklist - [ ] I have read the contributing guidelines From c9ee92988431cb1d83c4239d50b96d621edc5928 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 9 Aug 2024 13:43:23 +0000 Subject: [PATCH 089/155] docs(llamacpp): clarify upstream relicense and bug repro guidance --- src/instructlab/llamacpp/README.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/instructlab/llamacpp/README.md b/src/instructlab/llamacpp/README.md index 0eac638a58..7a55c83412 100644 --- a/src/instructlab/llamacpp/README.md +++ b/src/instructlab/llamacpp/README.md @@ -19,13 +19,17 @@ project. When updating files in this directory, please: against upstream remains easy to audit. 5. When pulling in a new upstream revision, re-check this `NOTICE` for any license or attribution updates that need to be mirrored here. +6. If upstream relicenses or adds third-party notices, mirror those + changes here in the same pull request that pulls in the new code. ## Reporting Issues Bugs that reproduce against an unmodified upstream build should be reported to the [llama.cpp issue tracker](https://github.com/ggerganov/llama.cpp/issues). File issues against `instructlab` only when the problem is specific to -the integration code in this directory. +the integration code in this directory. When in doubt, try to reproduce +against a stock `llama.cpp` build first and include those results in the +bug report so maintainers can quickly route the issue. ## Original License From 9354a74e0fd34d2fd4a59d89c272e971da47914a Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 14 Aug 2024 16:46:56 +0000 Subject: [PATCH 090/155] Tighten reporting-issues guidance wording in llamacpp NOTICE --- src/instructlab/llamacpp/README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/instructlab/llamacpp/README.md b/src/instructlab/llamacpp/README.md index 7a55c83412..7b0eb83b06 100644 --- a/src/instructlab/llamacpp/README.md +++ b/src/instructlab/llamacpp/README.md @@ -27,9 +27,10 @@ project. When updating files in this directory, please: Bugs that reproduce against an unmodified upstream build should be reported to the [llama.cpp issue tracker](https://github.com/ggerganov/llama.cpp/issues). File issues against `instructlab` only when the problem is specific to -the integration code in this directory. When in doubt, try to reproduce -against a stock `llama.cpp` build first and include those results in the -bug report so maintainers can quickly route the issue. +the integration code in this directory. When in doubt, reproduce +against a stock `llama.cpp` build first and include those results in +the bug report so maintainers can quickly route the issue to the +appropriate project. ## Original License From f64a226ba32a867a8f4c15b03f5f36bd74f8303c Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 20 Aug 2024 20:35:39 +0000 Subject: [PATCH 091/155] Expand CoC with standards, enforcement, and report follow-up sections --- CODE_OF_CONDUCT.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 1d3a4028fb..03a29912d2 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -2,6 +2,20 @@ This project adheres to the [InstructLab - Code of Conduct and Covenant](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md). By participating in this project, you are expected to uphold this code. +The sections below summarize how that shared Code of Conduct applies specifically to the `instructlab/cli` repository. The linked community document remains the authoritative source; this file exists to make day-to-day expectations easier to find for contributors working in this repo. + +## Our Standards + +In line with the shared Code of Conduct, contributors and maintainers are expected to: + +- be welcoming, patient, and respectful toward people of all backgrounds and experience levels, +- assume good faith and ask clarifying questions before assuming bad intent, +- give and gracefully accept constructive feedback during code review, +- focus discussion on technical merit rather than on individuals, and +- defer to maintainers for final decisions on contested issues, while feeling free to disagree respectfully. + +Behavior that is not acceptable includes harassment, personal attacks, discriminatory jokes or language, sustained disruption of discussions, and publishing others' private information without explicit permission. + ## Reporting Please report any unacceptable behavior in accordance with the reporting guidelines outlined in the linked Code of Conduct. @@ -19,6 +33,16 @@ Reports are handled confidentially by the InstructLab community maintainers. See If you are unsure whether something rises to the level of a Code of Conduct violation, you are still encouraged to reach out — maintainers would rather hear about a concern early than learn about it after it has escalated. +### What to expect after reporting + +After a report is received, you can generally expect the following: + +1. An acknowledgement from a maintainer that the report has been received. +2. A private review of the report by the maintainers responsible for Code of Conduct enforcement. +3. A follow-up that may include requests for additional information, a summary of any action taken, or an explanation if no action is considered necessary. + +Maintainers will make a reasonable effort to keep the reporter informed about the status of their report, while also respecting the privacy of everyone involved. + ## Scope This Code of Conduct applies within all project spaces — including, but not limited to: @@ -33,6 +57,17 @@ This Code of Conduct applies within all project spaces — including, but not li It also applies when an individual is officially representing the project in public spaces. +## Enforcement + +Maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful. Possible responses include, but are not limited to: + +- a private warning, +- a request for a public or private apology, +- temporary or permanent removal of comments, commits, code, issues, or other contributions, +- temporary or permanent bans from project spaces. + +Maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. + ## Acknowledgements This document is a thin wrapper around the shared [InstructLab Code of Conduct](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md); please refer to that document as the authoritative source if the two ever appear to disagree. From ce74a675a3aabfc7db9095bbd01876f228727124 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 21 Aug 2024 21:13:43 +0000 Subject: [PATCH 092/155] docs(habana): annotate perf-relevant Gaudi env vars --- docs/habana-gaudi.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/docs/habana-gaudi.md b/docs/habana-gaudi.md index 5c8fe8658c..87d1984186 100644 --- a/docs/habana-gaudi.md +++ b/docs/habana-gaudi.md @@ -151,7 +151,12 @@ pip install './instructlab[habana]' > **TIP** If `llama-cpp-python` fails to build with error ``unsupported instruction `vpdpbusd'``, then install with `CFLAGS="-mno-avx" pip install ...`. -Train environment (see [Habana runtime environment variables](https://docs.habana.ai/en/latest/PyTorch/Reference/Runtime_Flags.html) +### Performance-related environment variables + +Training throughput on Gaudi 2 is sensitive to a handful of runtime flags. The +following values have produced the best results in our testing; see the +[Habana runtime environment variables](https://docs.habana.ai/en/latest/PyTorch/Reference/Runtime_Flags.html) +reference for the full list. ```shell # environment variables for training @@ -159,10 +164,13 @@ export TSAN_OPTIONS='ignore_noninstrumented_modules=1' export TCMALLOC_LARGE_ALLOC_REPORT_THRESHOLD=7516192768 export LD_PRELOAD=/lib64/libtcmalloc.so -# work around race condition on systems with lots of cores +# work around race condition on systems with lots of cores; +# values between 8 and 16 are typically a good trade-off between +# data-loader throughput and stability. export OMP_NUM_THREADS=16 -# Gaudi configuration +# Gaudi configuration (eager mode + 4-stage pipeline gives the best +# step time for the current training script) export PT_HPU_LAZY_MODE=0 export PT_HPU_ENABLE_EAGER_CACHE=TRUE export PT_HPU_EAGER_4_STAGE_PIPELINE_ENABLE=TRUE From a91f7a9c9d2b00f28690fb323746c9a120c76e79 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 23 Aug 2024 01:13:49 +0000 Subject: [PATCH 093/155] docs(ci): document E2E runtime expectations and perf tips --- docs/ci.md | 47 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/docs/ci.md b/docs/ci.md index 35247e425c..ea8966f46e 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -39,6 +39,49 @@ At a high level, the E2E job performs the following steps: If any step fails, the job aborts and the logs from the failing step are available in the run summary. +## Performance considerations + +GPU runner time is the most expensive resource consumed by CI, so the E2E +job is tuned to finish in a predictable window rather than to maximize +coverage. A few practical notes for contributors and maintainers: + +### Expected runtimes + +| Phase | Typical duration | Notes | +| ----------------------------- | ---------------- | -------------------------------------- | +| Checkout and dependency setup | 2–4 min | Dominated by `pip` resolution. | +| Model download | 3–8 min | Cached between runs when possible. | +| Inference smoke test | 1–2 min | Short prompt, single generation. | +| Synthetic data generation | 5–10 min | Reduced taxonomy subset. | +| Training pass | 10–20 min | Few steps, small batch. | + +A healthy end-to-end run typically completes in 25–45 minutes. Runs that +exceed an hour usually indicate a regression — either in `ilab` itself or +in an upstream dependency — and are worth investigating. + +### Keeping runs fast + +- Prefer cache-friendly changes: avoid bumping pinned dependency versions + in unrelated PRs, since that invalidates the pip and model caches. +- When adding new E2E steps, gate expensive ones behind an input flag so + they can be skipped for routine verification runs. +- If you need to iterate on the workflow file itself, test against a + small branch first rather than re-running the full matrix repeatedly. + +### Diagnosing slow runs + +The job uploads timing information alongside the regular logs. To compare +a suspect run against a recent baseline: + +```shell +gh run list --workflow=e2e.yml --limit 10 +gh run view --log | grep -E '^(Run|##\[group\])' +``` + +Look for steps whose duration is significantly higher than the table +above. Common culprits are cold model caches, network hiccups during +download, and training step counts that were accidentally increased. + ## Accessibility The steps above rely on the GitHub Actions web UI. To make this workflow @@ -71,4 +114,6 @@ we treat accessibility bugs as regular bugs. If a CI job appears to be broken (for example, failing for reasons unrelated to the change under test), please open an issue in the [instructlab/instructlab](https://github.com/instructlab/instructlab/issues) -repository and include a link to the failing run. +repository and include a link to the failing run. When the issue is a +performance regression rather than an outright failure, please include +links to both the slow run and a recent fast run for comparison. From 007a8ebfc2c36d17a205ec0ccf91143645cf44ca Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 26 Aug 2024 17:21:00 +0000 Subject: [PATCH 094/155] docs: note GPU index selection in containerization guide --- docs/containerization.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/containerization.md b/docs/containerization.md index c05c24e510..b03b90c485 100644 --- a/docs/containerization.md +++ b/docs/containerization.md @@ -64,6 +64,8 @@ podman run --device nvidia.com/gpu=0 --security-opt=label=disable -it ``` To expose all available GPUs to the container, use `nvidia.com/gpu=all` instead. +Individual GPUs can also be selected by index (for example, `nvidia.com/gpu=1`) +to match the device names produced by `nvidia-ctk cdi list`. Voila! You now have a container with CUDA and GPUs enabled! From 00451032517797bbaff28f70d3d933d662e2a8a7 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 29 Aug 2024 13:05:49 +0000 Subject: [PATCH 095/155] Ask for alt text on screenshots in bug reports --- .github/ISSUE_TEMPLATE/bug_report.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index e5002e2922..5061a32f76 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -31,7 +31,8 @@ the exit code. --> **Screenshots or Logs** +file rather than pasting inline. If you include screenshots, please add +descriptive alt text so the content is accessible to screen-reader users. --> **Device Info (please complete the following information):** - Hardware Specs: [e.g. Apple M2 Pro Chip, 16 GB Memory, etc.] From 141484626436e0049c9c7cb276e40a2206dd910e Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 4 Sep 2024 18:37:06 +0000 Subject: [PATCH 096/155] docs: add Reporting new issues to TOC and note dup search --- TROUBLESHOOTING.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md index 616fe90bf6..22e75228e8 100644 --- a/TROUBLESHOOTING.md +++ b/TROUBLESHOOTING.md @@ -11,6 +11,7 @@ This document is for commonly found problems and their solutions when using `ila - [Data generation](#data-generation) - [Training](#training) - [Additional resources](#additional-resources) +- [Reporting new issues](#reporting-new-issues) ## `ilab` troubleshooting @@ -102,7 +103,7 @@ The training step is run with the `ilab train` command. This step trains the mod ## Reporting new issues -If the steps above did not resolve your problem, please open an issue on the [InstructLab issue tracker](https://github.com/instructlab/instructlab/issues). When filing a new report, include the following so maintainers can help you faster: +If the steps above did not resolve your problem, please open an issue on the [InstructLab issue tracker](https://github.com/instructlab/instructlab/issues). Before filing, please search existing issues to avoid duplicates. When filing a new report, include the following so maintainers can help you faster: - The exact `ilab` command you ran and its full output. - The output of `ilab system info` (or, if unavailable, your OS, architecture, Python version, and `ilab --version`). From 15372c60566a26ff6fe60d502ede6ef7bb7d0f39 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 18 Sep 2024 22:07:03 +0000 Subject: [PATCH 097/155] Expand additional context hint in bug report template --- .github/ISSUE_TEMPLATE/bug_report.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 5061a32f76..d27e4eb571 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -91,4 +91,5 @@ provide, the easier it will be to triage and reproduce the regression. **Additional context** +changes to your environment, related issues, workarounds you've tried, +or links to similar issues you found while searching. --> From c331ff6786b9e074b53e78dfb4cacfd2729a9579 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 2 Oct 2024 20:13:04 +0000 Subject: [PATCH 098/155] Trim redundant phrasing in feature request template --- .github/ISSUE_TEMPLATE/feature_request.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 454bd87f01..8762014143 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -20,4 +20,4 @@ assignees: '' **Additional context** - + From 030be8854f6eb433ace6a6d383d40e84befbbb5c Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 2 Oct 2024 20:23:01 +0000 Subject: [PATCH 099/155] docs(notebooks): clarify Kaggle workflow is unmaintained --- notebooks/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/notebooks/README.md b/notebooks/README.md index 9fd2d61a22..a2f91abadc 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -40,7 +40,7 @@ Once you have finished training and the output looks good, we encourage you to p ### Kaggle (unsupported and deprecated) -> **Note:** The Kaggle workflow is no longer actively maintained. Prefer Google Colab or local `ilab train` where possible. +> **Note:** The Kaggle workflow is no longer actively maintained and may break without notice. Prefer Google Colab or local `ilab train` where possible. Using a [Kaggle Notebook](https://github.com/instructlab/instructlab/blob/main/notebooks/Training_a_LoRA_With_Instruct_Lab.ipynb) and the NVIDIA P100 provided in the free tier, we will fine tune a LoRA. From 4d979eaf579386a7b915d46fd866127428871fb7 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 3 Oct 2024 15:46:00 +0000 Subject: [PATCH 100/155] Expand a11y prompt with ARIA and WCAG guidance --- .github/ISSUE_TEMPLATE/feature_request.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 8762014143..2693ebddee 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -17,7 +17,7 @@ assignees: '' **Accessibility considerations** - + **Additional context** From 0daaf0a6993be9852edae28fe57ca4e3a343dbac Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 25 Oct 2024 14:43:43 +0000 Subject: [PATCH 101/155] docs(stable): add accessibility notes section --- docs/STABLE.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/STABLE.md b/docs/STABLE.md index 3ae88c4bf4..8d4daf4ec2 100644 --- a/docs/STABLE.md +++ b/docs/STABLE.md @@ -68,3 +68,13 @@ $ git push -f upstream stable ``` Finally, check the tags on the GitHub web UI to ensure everything looks correct. + +Accessibility notes for this document +------------------------------------- + +When following or updating these instructions, keep the following accessibility considerations in mind: + +- Each code block is labeled with the `ShellSession` language hint so screen readers and syntax highlighters can announce the content type clearly. +- Commands are written one per line, with the `$` prompt prefix, so users relying on line-by-line navigation can read each step independently. +- Links use descriptive text (for example, "tags" rather than "click here") so that users navigating by link list can understand the destination out of context. +- When referring to flags such as `-f` or `--force`, both the short and long forms are spelled out at least once to aid users who may not recognize a single-character flag in isolation. From 5d902d9c4330810ca98fd2773f3dcfbdf12e3fd1 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 30 Oct 2024 16:44:42 +0000 Subject: [PATCH 102/155] docs: note when to regenerate the CDI spec --- docs/containerization.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/containerization.md b/docs/containerization.md index b03b90c485..645a642d36 100644 --- a/docs/containerization.md +++ b/docs/containerization.md @@ -57,6 +57,9 @@ nvidia.com/gpu=0 nvidia.com/gpu=all ``` +The CDI spec should be regenerated any time the host's GPU drivers or hardware +change, otherwise `podman run` may fail to locate the devices. + ### 4. Run the container with GPU access ```shell From bcc375217334711d22b380ad35669903d83c3840 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sat, 2 Nov 2024 19:08:19 +0000 Subject: [PATCH 103/155] docs: name the license inline in mlx_explore README --- src/instructlab/mlx_explore/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/instructlab/mlx_explore/README.md b/src/instructlab/mlx_explore/README.md index 30772b99de..09a32096fc 100644 --- a/src/instructlab/mlx_explore/README.md +++ b/src/instructlab/mlx_explore/README.md @@ -2,7 +2,7 @@ The code in this folder is modified from the [mlx-examples](https://github.com/ml-explore/mlx-examples) repository on GitHub. For the original code, see [mlx-examples/llms/gguf_llm](https://github.com/ml-explore/mlx-examples/tree/main/llms/gguf_llm), -which is distributed under the following license. +which is distributed under the MIT License reproduced below. ```license MIT License From 1c9606d1ca21795887a1898ad9e476a5e7779cf3 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 5 Nov 2024 00:18:16 +0000 Subject: [PATCH 104/155] docs(habana): note OMP_NUM_THREADS fallback for load-time segfaults --- docs/habana-gaudi.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/habana-gaudi.md b/docs/habana-gaudi.md index 87d1984186..eb65fae97a 100644 --- a/docs/habana-gaudi.md +++ b/docs/habana-gaudi.md @@ -164,8 +164,9 @@ export TSAN_OPTIONS='ignore_noninstrumented_modules=1' export TCMALLOC_LARGE_ALLOC_REPORT_THRESHOLD=7516192768 export LD_PRELOAD=/lib64/libtcmalloc.so -# work around race condition on systems with lots of cores; -# values between 8 and 16 are typically a good trade-off between +# work around race condition on systems with lots of cores; if training +# still segfaults at model-load time, lower this further (down to 1). +# Values between 8 and 16 are typically a good trade-off between # data-loader throughput and stability. export OMP_NUM_THREADS=16 From 440c9b3072e18ec0840e0296ddd4071577f90484 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 6 Nov 2024 15:44:45 +0000 Subject: [PATCH 105/155] docs: clarify repo scope and triage info in MAINTAINERS.md --- MAINTAINERS.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/MAINTAINERS.md b/MAINTAINERS.md index 864f8ba8f8..5deb4e4e4d 100644 --- a/MAINTAINERS.md +++ b/MAINTAINERS.md @@ -2,6 +2,10 @@ For a complete list of InstructLab project Maintainers, see the [Maintainers list](https://github.com/instructlab/community/blob/main/MAINTAINERS.md) in the `instructlab/community` repository. +## Repository Scope + +This repository (`instructlab/sdg`) contains the Synthetic Data Generation (SDG) library used by InstructLab. Maintainers listed in the central community repository are responsible for reviewing and merging contributions here. + ## Reporting Issues If you need to contact the maintainers of this repository, please: @@ -10,6 +14,8 @@ If you need to contact the maintainers of this repository, please: - Start a [discussion](https://github.com/instructlab/sdg/discussions) for general questions. - Refer to [SECURITY.md](./SECURITY.md) for reporting security vulnerabilities. +When filing a bug report, please include the SDG version, Python version, and a minimal reproducer where possible. This helps maintainers triage performance and correctness issues more quickly. + ## Governance Project governance, including the process for becoming a maintainer, is documented in the [instructlab/community](https://github.com/instructlab/community) repository. Please review the governance documents before opening maintainer-related proposals. From daa45a51cc899bd539b2393285316e6b690f22b0 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 20 Nov 2024 22:56:06 +0000 Subject: [PATCH 106/155] docs: show explicit command to reset iogpu.wired_limit_mb --- TROUBLESHOOTING.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md index 22e75228e8..8224d0c18a 100644 --- a/TROUBLESHOOTING.md +++ b/TROUBLESHOOTING.md @@ -35,7 +35,11 @@ For example, on a M1 with 16GB RAM, the `ilab generate` command with the limit b sudo sysctl iogpu.wired_limit_mb=12288 ``` -Once done, make sure to reset the limit back to 0, which is the default. +Once done, make sure to reset the limit back to 0, which is the default: + +```shell +sudo sysctl iogpu.wired_limit_mb=0 +``` > **Note:** This value will reset to the default after the machine reboots. From c5df630842b257311a4eea8319d3acfdb483ddbc Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 21 Nov 2024 20:14:49 +0000 Subject: [PATCH 107/155] Add minimal reproducer section to bug report template --- .github/ISSUE_TEMPLATE/bug_report.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index d27e4eb571..6432939d18 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -54,6 +54,12 @@ secrets removed) and note any non-default settings. --> of InstructLab and, if possible, the first version where you observed the bug. --> +**Minimal reproducer** + + **Performance (if applicable)** **Accessibility considerations** - + **Additional context** From 640e8dfaedfaf8019d6adce9c1bf1e4b349e8fe0 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 21 Jan 2025 03:15:01 +0000 Subject: [PATCH 118/155] Document `ilab` command-not-found and OOM troubleshooting --- TROUBLESHOOTING.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/TROUBLESHOOTING.md b/TROUBLESHOOTING.md index 8224d0c18a..5873321dd6 100644 --- a/TROUBLESHOOTING.md +++ b/TROUBLESHOOTING.md @@ -6,6 +6,8 @@ This document is for commonly found problems and their solutions when using `ila - [`ilab` troubleshooting](#ilab-troubleshooting) - [`ilab generate` command running slow on macOS](#ilab-generate-command-running-slow-on-macos) + - [`ilab` command not found after install](#ilab-command-not-found-after-install) + - [Out-of-memory (OOM) errors during training](#out-of-memory-oom-errors-during-training) - [Model fine tuning and response optimization](#model-fine-tuning-and-response-optimization) - [Skill composition](#skill-composition) - [Data generation](#data-generation) @@ -43,6 +45,36 @@ sudo sysctl iogpu.wired_limit_mb=0 > **Note:** This value will reset to the default after the machine reboots. +### `ilab` command not found after install + +If your shell reports `ilab: command not found` immediately after a successful `pip install`, the most common cause is that the Python `bin/` directory associated with your virtual environment (or user site-packages) is not on your `PATH`. + +1. Confirm that the package is installed and locate its entry point: + + ```shell + python -m pip show instructlab + python -c "import shutil; print(shutil.which('ilab'))" + ``` + +2. If the second command prints `None`, activate the virtual environment you installed into (for example `source venv/bin/activate`) and try again. When installing with `pip install --user`, ensure that `python -m site --user-base` followed by `/bin` is on your `PATH`. + +3. As a workaround, you can always invoke the CLI through the module form: + + ```shell + python -m instructlab --help + ``` + +### Out-of-memory (OOM) errors during training + +`ilab train` can fail with an OOM error on systems that are close to the minimum supported memory. If you see messages such as `CUDA out of memory`, `MPS backend out of memory`, or the process is killed by the OS, try the following before opening an issue: + +1. Close other memory-heavy applications (browsers, IDEs, container runtimes) and re-run the command. +2. Reduce the batch size or the number of training iterations passed to `ilab train`. +3. On macOS, see the [section above](#ilab-generate-command-running-slow-on-macos) for adjusting `iogpu.wired_limit_mb`. +4. On Linux with NVIDIA GPUs, verify available VRAM with `nvidia-smi` immediately before launching training; a stale process from a previous run may still be holding memory. + +If the failure persists even with a reduced workload, capture the full command output and include it in your bug report. + ## Model fine tuning and response optimization If you are looking to optimize the quality of the outputs generated by the model, there are a number of steps and parameters at various stages of the CLI workflow that you can consider leveraging. Some of these steps are discussed in the following sections. From b381c68fc2b4c614f3d246bf99ad4765d4370362 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sun, 26 Jan 2025 18:44:26 +0000 Subject: [PATCH 119/155] Document conflict-of-interest recusal in CoC enforcement --- CODE_OF_CONDUCT.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index bbcc8c090d..074dfddf28 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -68,6 +68,10 @@ Maintainers are responsible for clarifying the standards of acceptable behavior Enforcement decisions should be proportionate to the severity and pattern of the behavior, and maintainers will aim to apply them consistently across contributors. Maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. +### Conflicts of interest + +If a report involves a maintainer, or someone with whom a maintainer has a close personal or professional relationship, that maintainer is expected to recuse themselves from the review and decision-making process for the report. The remaining maintainers will handle the report, and may consult with the broader InstructLab community leadership if additional perspective is needed to reach a fair outcome. + ## Acknowledgements This document is a thin wrapper around the shared [InstructLab Code of Conduct](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md); please refer to that document as the authoritative source if the two ever appear to disagree. From f6836d9293d6b90728a0d2c0bc3bc3e4902ffc11 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sat, 1 Feb 2025 16:54:48 +0000 Subject: [PATCH 120/155] docs(rocm): clarify id -nG usage for checking group membership --- containers/rocm/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/containers/rocm/README.md b/containers/rocm/README.md index 3201c3670c..178c64db0c 100644 --- a/containers/rocm/README.md +++ b/containers/rocm/README.md @@ -32,7 +32,7 @@ You can verify that the GPU is visible inside the toolbox by running `rocminfo | grep -E 'Name|gfx'` and confirming that your card's `gfx` target is listed as an agent. If no agent is reported, double check that `/dev/kfd` and `/dev/dri/renderD*` exist on the host and that your user is in the -`render` and `video` groups (`id -nG`). +`render` and `video` groups (run `id -nG` to list your current groups). ## Building for other GPU architectures From 3425e4f69d0b3ee17efa2b739eef7a4241aa7fef Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 4 Feb 2025 18:40:34 +0000 Subject: [PATCH 121/155] Document accessibility expectations for governance processes --- governance.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/governance.md b/governance.md index 926ee8f567..cac04e68ab 100644 --- a/governance.md +++ b/governance.md @@ -42,6 +42,19 @@ Governance-related concerns, including questions about maintainer conduct or pro When reporting, please include enough context (links to issues, pull requests, or discussions) to allow the recipients to understand the situation, but avoid sharing private information about other contributors without their consent. If in doubt, redact identifying details and let the recipients ask follow-up questions privately. +If any of the channels above present an accessibility barrier — for example, if screen-reader navigation of GitHub issues is impractical for you, or if a synchronous community meeting is not workable — please reach out by email to the Oversight Committee and we will arrange an alternative intake path. No contributor should be required to use a specific tool in order to raise a governance concern. + +## Accessibility of Governance Processes + +The project aims to keep governance participation accessible to contributors with a wide range of abilities, time zones, and communication preferences. In practice, this means: + +- Governance discussions of record happen in writing (issues, pull requests, and mailing list threads) so that contributors who cannot attend synchronous meetings, or who rely on assistive technology, can participate on equal footing. +- Community meeting notes and recordings, when available, are linked from the relevant issue so that decisions are not effectively gated on live attendance. +- Maintainers are expected to summarize out-of-band conversations (video calls, chat threads) back into the relevant issue or pull request before a decision is finalized. +- Documents under this repository should follow basic accessibility practices: descriptive link text rather than "click here", alt text on images and diagrams, and heading levels used in order so that screen readers can navigate the structure. + +If you notice a governance process that is hard to participate in for accessibility reasons, please open an issue (or contact a maintainer privately) so we can adjust it. + ## Amendments This document may be updated over time to reflect changes in project practice. Substantive changes should be proposed via pull request and require approval from a majority of active maintainers before being merged. Minor edits (typos, formatting, broken links) may be merged under the normal review process. From 1f5dffc864d914e4d0b96fa4310a0a84024905d9 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 5 Feb 2025 18:50:16 +0000 Subject: [PATCH 122/155] docs(notebooks): fix NVIDIA capitalization and VRAM unit spacing --- notebooks/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/notebooks/README.md b/notebooks/README.md index 5e6b9e605e..e1b52a7e37 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -32,7 +32,7 @@ We've laid out the steps to get started with either platform below. Pre-requisites: * [Google Colab](https://research.google.com/colaboratory/faq.html) -* A Gmail account that you're logged into. This will allow you to use Google Colab, which in the free tier will give you access to an NVidia T4 with 15GB of VRAM. +* A Gmail account that you're logged into. This will allow you to use Google Colab, which in the free tier will give you access to an NVIDIA T4 with 15 GB of VRAM. ## Running the notebook From 9761af3ffd477d975f4e787617139ed88461def3 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 6 Feb 2025 15:27:38 +0000 Subject: [PATCH 123/155] docs: clarify upstream repo link in lora/models NOTICE --- src/instructlab/train/lora_mlx/models/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/instructlab/train/lora_mlx/models/README.md b/src/instructlab/train/lora_mlx/models/README.md index 5dafd789de..73034e280a 100644 --- a/src/instructlab/train/lora_mlx/models/README.md +++ b/src/instructlab/train/lora_mlx/models/README.md @@ -1,6 +1,6 @@ # Notice -The code in this folder is modified from the [mlx-examples lora/models](https://github.com/ml-explore/mlx-examples/tree/main/lora/models) directory (upstream commit history preserved in git blame). It is distributed under the MIT License, reproduced below. +The code in this folder is modified from the [mlx-examples lora/models](https://github.com/ml-explore/mlx-examples/tree/main/lora/models) directory in the [ml-explore/mlx-examples](https://github.com/ml-explore/mlx-examples) repository (upstream commit history preserved in git blame). It is distributed under the MIT License, reproduced below. ```license MIT License From 9db4abfac90dad719bc2a81a3783ce97fe7ad34a Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 12 Feb 2025 01:25:48 +0000 Subject: [PATCH 124/155] docs: clarify license text is reproduced for reference --- src/instructlab/generator/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/instructlab/generator/README.md b/src/instructlab/generator/README.md index 967dfc3ef3..6780f78b8e 100644 --- a/src/instructlab/generator/README.md +++ b/src/instructlab/generator/README.md @@ -1,6 +1,6 @@ ### Notice -This folder uses code from the [`stanford_alpaca`](https://github.com/tatsu-lab/stanford_alpaca) project developed by Tatsu Lab. The original code is licensed under the Apache License, Version 2.0: +This folder uses code from the [`stanford_alpaca`](https://github.com/tatsu-lab/stanford_alpaca) project developed by Tatsu Lab. The original code is licensed under the Apache License, Version 2.0, reproduced below for reference: ```license Apache License From ac97a39b78df1a2e519674ed3bd0704ab2ff0f01 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 12 Feb 2025 16:17:39 +0000 Subject: [PATCH 125/155] Clarify that accessibility reports are treated as process bugs --- governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/governance.md b/governance.md index cac04e68ab..deca55499d 100644 --- a/governance.md +++ b/governance.md @@ -53,7 +53,7 @@ The project aims to keep governance participation accessible to contributors wit - Maintainers are expected to summarize out-of-band conversations (video calls, chat threads) back into the relevant issue or pull request before a decision is finalized. - Documents under this repository should follow basic accessibility practices: descriptive link text rather than "click here", alt text on images and diagrams, and heading levels used in order so that screen readers can navigate the structure. -If you notice a governance process that is hard to participate in for accessibility reasons, please open an issue (or contact a maintainer privately) so we can adjust it. +If you notice a governance process that is hard to participate in for accessibility reasons, please open an issue (or contact a maintainer privately) so we can adjust it. Reports of this kind are treated as bugs in the governance process, not as personal complaints. ## Amendments From 88937501efe0beaa8571950876728b78c736c03e Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 14 Feb 2025 02:45:34 +0000 Subject: [PATCH 126/155] Add closing note to SECURITY.md --- SECURITY.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/SECURITY.md b/SECURITY.md index 705bb4206e..e4548e9189 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -3,3 +3,5 @@ You can find information on how to report a potential security vulnerability, as well as where to subscribe to receive security alerts, on the InstructLab project's [Security Page](https://github.com/instructlab/community/blob/main/SECURITY.md). Please do not report security vulnerabilities through public GitHub issues, discussions, or pull requests. + +Thank you for helping keep InstructLab and its users safe. From 903e4e618755407d136d05caa789ab4f54841fa8 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 18 Feb 2025 03:09:44 +0000 Subject: [PATCH 127/155] Tighten wording in Acknowledgements section --- CODE_OF_CONDUCT.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 074dfddf28..2d19feecbd 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -74,4 +74,4 @@ If a report involves a maintainer, or someone with whom a maintainer has a close ## Acknowledgements -This document is a thin wrapper around the shared [InstructLab Code of Conduct](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md); please refer to that document as the authoritative source if the two ever appear to disagree. +This document is a thin wrapper around the shared [InstructLab Code of Conduct](https://github.com/instructlab/community/blob/main/CODE_OF_CONDUCT.md); please refer to that document as the authoritative source if the two ever disagree. From 99c6bb8e326474c2dbd47d0e9c2d96a121d4dcd6 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 20 Feb 2025 18:00:39 +0000 Subject: [PATCH 128/155] Ask for profiler output and swap/memory-pressure info in perf bugs --- .github/ISSUE_TEMPLATE/bug_report.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index f5f16d539a..10ef9ef4b7 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -82,6 +82,8 @@ provide, the easier it will be to triage and reproduce the regression. `rocm-smi`, `top`, or `htop` output). Sustained vs. bursty usage is helpful to note. - Disk I/O or network activity if the workload is I/O-bound. + - Swap usage or signs of memory pressure (e.g. OOM-killer messages + in `dmesg`), which can masquerade as a pure CPU/GPU slowdown. Workload shape: - Dataset / model size and any batch size, sequence length, or @@ -89,6 +91,13 @@ provide, the easier it will be to triage and reproduce the regression. - Number of GPUs / workers and any relevant distributed settings. - Quantization, precision (fp16/bf16/fp32), or other tuning flags. + Profiling data (optional but very helpful): + - Any profiler output you have collected (e.g. `py-spy`, `perf`, + `torch.profiler`, Nsight Systems traces). A flamegraph or the + top few hot frames is often enough to point at a culprit. + - For GPU workloads, a short `nvidia-smi dmon` or `rocm-smi` + sample covering the slow phase. + Reproducibility: - Whether the slowdown is reproducible on a fresh environment. - Whether it occurs consistently or only intermittently, and how From 61eaccdeaebff3635d50c727bac7ac4a535be169 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 20 Feb 2025 22:22:32 +0000 Subject: [PATCH 129/155] docs: remind contributors to reactivate venv in new shells --- CONTRIBUTING/CONTRIBUTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTING/CONTRIBUTING.md b/CONTRIBUTING/CONTRIBUTING.md index 9e9feb4e47..d63a5df2a3 100644 --- a/CONTRIBUTING/CONTRIBUTING.md +++ b/CONTRIBUTING/CONTRIBUTING.md @@ -70,7 +70,7 @@ The following tools are required: - [`coreutils`](https://www.gnu.org/software/coreutils/) (for functional tests) - [`bash`](https://www.gnu.org/software/bash/) (v5+, for functional tests) -It is recommended to work inside a Python [virtual environment](https://docs.python.org/3/library/venv.html) (e.g., `python -m venv venv && source venv/bin/activate`) to keep project dependencies isolated from your system Python. +It is recommended to work inside a Python [virtual environment](https://docs.python.org/3/library/venv.html) (e.g., `python -m venv venv && source venv/bin/activate`) to keep project dependencies isolated from your system Python. Remember to activate the same virtual environment in any new shell before running `tox` or `pip` commands below. You can setup your dev environment using [`tox`](https://tox.wiki/en/latest/), an environment orchestrator which allows for setting up environments for and invoking builds, unit tests, formatting, linting, etc. Install tox with: From 102bdd64789c6c481b0073e400035d2ebd9116f8 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sat, 22 Feb 2025 17:28:58 +0000 Subject: [PATCH 130/155] docs(ci): note 20% wall-time threshold for regressions --- docs/ci.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/ci.md b/docs/ci.md index ea8966f46e..1572c55d84 100644 --- a/docs/ci.md +++ b/docs/ci.md @@ -57,7 +57,9 @@ coverage. A few practical notes for contributors and maintainers: A healthy end-to-end run typically completes in 25–45 minutes. Runs that exceed an hour usually indicate a regression — either in `ilab` itself or -in an upstream dependency — and are worth investigating. +in an upstream dependency — and are worth investigating. As a rough rule +of thumb, a sustained >20% increase in total wall time across several +consecutive runs on `main` is worth a closer look. ### Keeping runs fast From 05be24e4a35f18b12f35620afded3680ef3eeebd Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 24 Feb 2025 16:50:23 +0000 Subject: [PATCH 131/155] Note community repo as source of truth in roles doc --- CONTRIBUTOR_ROLES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CONTRIBUTOR_ROLES.md b/CONTRIBUTOR_ROLES.md index f5a9d2ee70..ab57b1276a 100644 --- a/CONTRIBUTOR_ROLES.md +++ b/CONTRIBUTOR_ROLES.md @@ -8,4 +8,4 @@ For information about contributor roles, including the responsibilities and requ - **Approvers** can approve pull requests for merge in their areas of expertise. - **Reviewers** can review pull requests and provide feedback, but cannot approve them for merge. -For the full, authoritative definitions and the process for advancing between roles, always refer to the community repository linked above. +For the full, authoritative definitions and the process for advancing between roles, always refer to the community repository linked above. The community repository is the single source of truth; this document is a convenience summary and may lag behind. From 1d051c08af9569e407b13ff9a4f3e0515f356f1f Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 25 Feb 2025 23:05:33 +0000 Subject: [PATCH 132/155] docs: add performance tuning section to containerization guide --- docs/containerization.md | 64 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/docs/containerization.md b/docs/containerization.md index 9634cf45da..6e24fd42fe 100644 --- a/docs/containerization.md +++ b/docs/containerization.md @@ -86,6 +86,68 @@ the command is not found or no devices are listed, double-check that the CDI specification was generated on the host and that the correct device name was passed to `podman run`. +## Performance tuning + +The defaults produced by the steps above are functional, but a handful of +adjustments tend to make `ilab train` and `ilab generate` noticeably faster on +multi-GPU hosts. None of these are required, but they are worth experimenting +with if throughput is lower than you expect. + +### Pin GPUs explicitly + +When running on a shared host, prefer pinning specific GPUs over +`nvidia.com/gpu=all`. This avoids contention with other workloads and makes +benchmarks reproducible: + +```shell +podman run \ + --device nvidia.com/gpu=0 \ + --device nvidia.com/gpu=1 \ + --security-opt=label=disable \ + -it +``` + +Use `nvidia-smi topo -m` on the host to pick GPUs that share an NVLink or PCIe +switch — colocated devices typically yield better all-reduce performance during +training. + +### Give the container enough shared memory + +The default 64 MB `/dev/shm` allocation is far too small for PyTorch data +loaders with multiple workers and frequently shows up as mysterious hangs or +`Bus error` crashes. Bump it to at least a few gigabytes: + +```shell +podman run --shm-size=8g --device nvidia.com/gpu=all \ + --security-opt=label=disable -it +``` + +### Use a persistent cache volume + +Model weights, tokenizers, and compiled CUDA kernels are expensive to download +and rebuild on every container start. Mounting a host directory into the +container keeps them across runs: + +```shell +podman run \ + -v $HOME/.cache/instructlab:/root/.cache/instructlab:Z \ + -v $HOME/.cache/huggingface:/root/.cache/huggingface:Z \ + --device nvidia.com/gpu=all --security-opt=label=disable \ + --shm-size=8g -it +``` + +### Confirm the expected runtime is active + +A quick sanity check inside the container: + +```shell +nvidia-smi --query-gpu=name,driver_version,memory.total --format=csv +python -c "import torch; print(torch.cuda.is_available(), torch.cuda.device_count())" +``` + +If `torch.cuda.is_available()` returns `False`, the container is silently +falling back to CPU and any performance numbers you collect will be misleading. + ## Troubleshooting - **`nvidia-ctk: command not found`** — the toolkit was not installed. Re-run @@ -99,6 +161,8 @@ passed to `podman run`. - **`nvidia-smi` reports a driver/library version mismatch** — the host driver is older than the CUDA runtime inside the image. Update the host driver or rebuild the image against an older CUDA base. +- **Data loader hangs or `Bus error` during training** — `/dev/shm` is too + small; rerun with `--shm-size=8g` (or larger) as described above. ### Sources From 17986bf06855626e61a4c8295fa7c390c70b4f5d Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 27 Feb 2025 04:42:34 +0000 Subject: [PATCH 133/155] Note recusals in internal report record --- CODE_OF_CONDUCT.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md index 2d19feecbd..29e089a673 100644 --- a/CODE_OF_CONDUCT.md +++ b/CODE_OF_CONDUCT.md @@ -70,7 +70,7 @@ Enforcement decisions should be proportionate to the severity and pattern of the ### Conflicts of interest -If a report involves a maintainer, or someone with whom a maintainer has a close personal or professional relationship, that maintainer is expected to recuse themselves from the review and decision-making process for the report. The remaining maintainers will handle the report, and may consult with the broader InstructLab community leadership if additional perspective is needed to reach a fair outcome. +If a report involves a maintainer, or someone with whom a maintainer has a close personal or professional relationship, that maintainer is expected to recuse themselves from the review and decision-making process for the report. The remaining maintainers will handle the report, and may consult with the broader InstructLab community leadership if additional perspective is needed to reach a fair outcome. Recusal should be noted in the internal record of the report so that it is clear who participated in the decision. ## Acknowledgements From fa0b93a3d95ac582dd31c683d3814f5959f54ca6 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 28 Feb 2025 22:29:21 +0000 Subject: [PATCH 134/155] docs(gguf): document --outtype options and Q2_K row --- docs/converting_GGUF.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/docs/converting_GGUF.md b/docs/converting_GGUF.md index 461c1f5def..cd60c532e8 100644 --- a/docs/converting_GGUF.md +++ b/docs/converting_GGUF.md @@ -65,6 +65,20 @@ python convert-hf-to-gguf.py $MODEL_DIR --outtype f16 > Note: This may take about a minute or so. +### Choosing an `--outtype` + +The `--outtype` flag controls the precision of the resulting GGUF file: + +- `f32` — full 32-bit floats. Largest output, no precision loss relative + to the source weights. Useful primarily for debugging. +- `f16` — 16-bit floats. The most common starting point before + quantization, and roughly half the size of `f32`. +- `q8_0` — convert directly to an 8-bit quantized GGUF. Handy when you + are short on disk space and do not need an intermediate `f16` file. + +If you intend to quantize further (see below), `f16` is usually the +right choice. + ## Quantize Optionally, for smaller/faster models with varying loss of quality use a @@ -106,6 +120,10 @@ types produce smaller and faster models at the cost of some output quality. | `Q5_K_M` | ~5 | Good balance between size and quality | | `Q4_K_M` | ~4 | Popular default; smaller with modest quality loss | | `Q3_K_M` | ~3 | Smaller still, more noticeable quality loss | +| `Q2_K` | ~2 | Smallest; quality degradation is often visible | + +As a rough rule of thumb, a 7B-parameter model is around 13 GB at `f16`, +~7 GB at `Q8_0`, ~4 GB at `Q4_K_M`, and ~3 GB at `Q3_K_M`. ## Verifying the converted model @@ -128,3 +146,6 @@ be used with downstream tooling. directory. - **Out-of-memory while converting:** try converting on a machine with more RAM, or convert directly to a lower precision via `--outtype`. +- **Garbled or repetitive output after quantization:** the quantization + level may be too aggressive for the model. Try a higher-precision type + (for example, step up from `Q3_K_M` to `Q4_K_M` or `Q5_K_M`). From 574a22c5b20bff731830744c0dc47f8ce2c37d54 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 5 Mar 2025 16:04:57 +0000 Subject: [PATCH 135/155] governance: document voluntary maintainer step-down process --- governance.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/governance.md b/governance.md index deca55499d..3b381b652f 100644 --- a/governance.md +++ b/governance.md @@ -32,6 +32,10 @@ Nominations may be made by any existing maintainer by opening a pull request tha Maintainers who become inactive for an extended period (typically six months without review, commit, or issue activity) may be moved to an emeritus list by the same process. Emeritus maintainers are welcome to return to active status at any time by request. +## Stepping Down + +Maintainers who wish to step down voluntarily are encouraged to do so by opening a pull request that moves their entry from the active maintainer list to the emeritus list. Where practical, please give a short heads-up to the other maintainers (in an issue, on the mailing list, or in chat) so that ongoing reviews and triage responsibilities can be handed off cleanly. There is no expectation of justification or notice period — stepping down is a normal part of a healthy project, and returning later is always welcome. + ## Reporting Concerns Governance-related concerns, including questions about maintainer conduct or project direction, can be raised by: From f70dddf7bafa0669e21422bc86d3bcdcbb0ab53e Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sat, 8 Mar 2025 19:10:03 +0000 Subject: [PATCH 136/155] docs(gpu): add one-liner rebuild template for llama-cpp backends --- docs/gpu-acceleration.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/docs/gpu-acceleration.md b/docs/gpu-acceleration.md index 7623984364..75652b0cbd 100644 --- a/docs/gpu-acceleration.md +++ b/docs/gpu-acceleration.md @@ -29,6 +29,23 @@ the examples is `0.2.55`. If you bump that version, remember to clear the pip wheel cache (`pip cache remove llama_cpp_python`) before reinstalling so the rebuild actually happens. +### One-liner rebuild template + +Every backend section below boils down to the same two-step recipe. If you +already know which flag you need, you can skip ahead and just run: + +```shell +export LLAMA_CPP_VERSION=0.2.55 +export BACKEND=CUBLAS # or HIPBLAS, METAL, CLBLAST + +pip cache remove llama_cpp_python +pip install --force-reinstall "llama_cpp_python==${LLAMA_CPP_VERSION}" \ + -C cmake.args="-DLLAMA_${BACKEND}=on" +``` + +The per-backend sections below add the extra system packages, driver checks, +and environment variables required to make that rebuild actually succeed. + ## Python 3.11 (Linux only) > **NOTE:** This section may be outdated. At least AMD ROCm works fine with From f5608fca06829148337f9c8b5e6a817ec93cd764 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sun, 9 Mar 2025 15:14:56 +0000 Subject: [PATCH 137/155] Expand PR template with related work, repro steps, and compat sections --- .github/PULL_REQUEST_TEMPLATE.md | 37 ++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index df72e56faa..e5d851989e 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -24,6 +24,19 @@ have, such as design decisions, trade-offs, or alternatives considered. - [ ] Documentation update - [ ] Refactor / code cleanup - [ ] Build, CI, or tooling change +- [ ] Performance improvement +- [ ] Test-only change + +## Related Work + + + +- Related issues: +- Related PRs: +- Design docs / RFCs: ## Testing @@ -36,6 +49,14 @@ reviewers can reproduce. Include relevant details about your test setup. - [ ] New tests have been added to cover the changes - [ ] Manual verification was performed (describe below) +### Reproduction Steps + + + +1. +2. +3. + ## Risk Assessment +- **Blast radius:** +- **Rollback plan:** +- **Migrations required:** + +## Backwards Compatibility + + + +- [ ] Fully backwards compatible +- [ ] Backwards compatible with deprecation warnings +- [ ] Breaking change (migration notes below) + ## Checklist - [ ] I have read the contributing guidelines @@ -52,6 +88,7 @@ incompatibility, performance-sensitive paths, or security implications. - [ ] Tests added or updated - [ ] Documentation updated (if applicable) - [ ] No new warnings or errors have been introduced +- [ ] Public API changes are documented in the changelog ## Screenshots / Additional Notes From bd1b620c99f046098210eb0f6264afa441255ef9 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 11 Mar 2025 21:13:56 +0000 Subject: [PATCH 138/155] Clarify scope of minor edits in Amendments section --- governance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/governance.md b/governance.md index 3b381b652f..6fe823a00a 100644 --- a/governance.md +++ b/governance.md @@ -61,4 +61,4 @@ If you notice a governance process that is hard to participate in for accessibil ## Amendments -This document may be updated over time to reflect changes in project practice. Substantive changes should be proposed via pull request and require approval from a majority of active maintainers before being merged. Minor edits (typos, formatting, broken links) may be merged under the normal review process. +This document may be updated over time to reflect changes in project practice. Substantive changes should be proposed via pull request and require approval from a majority of active maintainers before being merged. Minor edits (typos, formatting, and broken or outdated links) may be merged under the normal review process without a separate announcement. From 1b728d815c202223ac41eff10fe325ea11de6a75 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 12 Mar 2025 16:48:29 +0000 Subject: [PATCH 139/155] Add network environment and submission checklist to bug report --- .github/ISSUE_TEMPLATE/bug_report.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 10ef9ef4b7..5c5c601907 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -60,6 +60,16 @@ shortest command, script, or taxonomy snippet that triggers the bug on a fresh checkout. Reproducers that do not require private data or special hardware are especially helpful for triage. --> +**Network environment (if applicable)** + + **Performance (if applicable)** +**Checklist** + + **Additional context** -# Knowledge +# Knowledge -Knowledge is an awareness of facts, a familiarity with individuals and situations, -or a practical skill. Knowledge of facts, also called propositional knowledge, is often characterized -as true belief that is distinct from opinion or guesswork by virtue of justification. -While there is wide agreement among philosophers that propositional knowledge is a form of true belief, -many controversies focus on justification. This includes questions like how to understand justification, -whether it is needed at all, and whether something else besides it is needed. -These controversies intensified in the latter half of the 20 century due to a series of thought experiments -called Get tier cases that provoked alternative definitions. +Knowledge is an awareness of facts, a familiarity with individuals and situations, +or a practical skill. Knowledge of facts, also called propositional knowledge, is often characterized +as true belief that is distinct from opinion or guesswork by virtue of justification. +While there is wide agreement among philosophers that propositional knowledge is a form of true belief, +many controversies focus on justification. This includes questions like how to understand justification, +whether it is needed at all, and whether something else besides it is needed. +These controversies intensified in the latter half of the 20th century due to a series of thought experiments +called Gettier cases that provoked alternative definitions. From 78f198b5745e8613db9e42c93752ac9c134245f1 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 25 Mar 2025 22:30:15 +0000 Subject: [PATCH 145/155] docs: use sudo for nvidia-ctk cdi list to match generate step --- docs/containerization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/containerization.md b/docs/containerization.md index 6e24fd42fe..a8f2309aaf 100644 --- a/docs/containerization.md +++ b/docs/containerization.md @@ -46,7 +46,7 @@ sudo dnf install -y nvidia-container-toolkit ```shell sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml -nvidia-ctk cdi list +sudo nvidia-ctk cdi list ``` Example output: From 4607bb909d0d6b6424678fec9ae7e1805d3e23f1 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 2 Apr 2025 23:08:25 +0000 Subject: [PATCH 146/155] docs(rocm): quantify CPU fallback slowdown in group setup note --- containers/rocm/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/containers/rocm/README.md b/containers/rocm/README.md index 178c64db0c..96aad3ff86 100644 --- a/containers/rocm/README.md +++ b/containers/rocm/README.md @@ -20,7 +20,9 @@ The container has all Python dependencies installed in a virtual env. The virtua Note: after adding yourself to the `render` and `video` groups, you must log out and back in (or run `newgrp`) for the new group membership to take effect. Without that, the container will not have access to `/dev/kfd` and GPU -inference will silently fall back to CPU, which is dramatically slower. +inference will silently fall back to CPU, which is dramatically slower (often +10-50x). If training or generation feels unexpectedly sluggish, this is the +first thing to verify. To update InstructLab CLI to latest version: `pip install -e ~/path/to/instructlab/instructlab` From 8fdfcc07d8a31f5a6e88ff68d637fbb9bc59fa7e Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Wed, 9 Apr 2025 00:46:43 +0000 Subject: [PATCH 147/155] Add matcher for lowercase pylint info codes --- .github/workflows/matchers/pylint.json | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/.github/workflows/matchers/pylint.json b/.github/workflows/matchers/pylint.json index 4a35f24f3d..a3a72139a7 100644 --- a/.github/workflows/matchers/pylint.json +++ b/.github/workflows/matchers/pylint.json @@ -83,6 +83,20 @@ "code": 5 } ] + }, + { + "owner": "pylint-info-lowercase", + "severity": "notice", + "pattern": [ + { + "regexp": "^(.+):(\\d+):(\\d+):\\s((i\\d{4}):\\s.+)$", + "file": 1, + "line": 2, + "column": 3, + "message": 4, + "code": 5 + } + ] } ] } From 5990cf8153ed5caf365bea14a5c132c01363c954 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 15 Apr 2025 20:32:43 +0000 Subject: [PATCH 148/155] docs: document release validation checklist --- docs/release-strategy.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/release-strategy.md b/docs/release-strategy.md index 66671c9165..d709bdce2a 100644 --- a/docs/release-strategy.md +++ b/docs/release-strategy.md @@ -68,3 +68,19 @@ The following are the steps for how Y-stream and Z-stream releases get cut. - InstructLab Slack 8. Create a milestone on GitHub for the next release without a milestone. - For a Z-Stream release skip this step. + +## Release Validation + +Before moving the `stable` tag forward in step 6 above, the Release Manager +(or a delegate) is responsible for performing a baseline validation of the +candidate build. At a minimum, this should include: + +- Installing the tagged release from PyPI into a clean virtual environment. +- Running `ilab --version` and confirming the reported version matches the tag. +- Executing the end-to-end smoke flow (`ilab config init`, `ilab data generate`, + and a short `ilab model train` run) on at least one supported platform. +- Spot-checking that the published wheel and sdist artifacts on PyPI match the + GitHub release assets. + +If any of these checks fail, the Release Manager should hold the `stable` tag +at its previous value and coordinate a Z-stream fix. From 7a825b2d01cdb2c119d180b0bc9242148cff7499 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Sun, 20 Apr 2025 14:33:09 +0000 Subject: [PATCH 149/155] docs(lora_mlx): note Apple Silicon requirement in Setup --- src/instructlab/train/lora_mlx/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/instructlab/train/lora_mlx/README.md b/src/instructlab/train/lora_mlx/README.md index 3712c08c78..f6c8def0f7 100644 --- a/src/instructlab/train/lora_mlx/README.md +++ b/src/instructlab/train/lora_mlx/README.md @@ -30,6 +30,9 @@ Install the dependencies: pip install -r requirements.txt ``` +Note that this example requires Apple Silicon (M1 or newer) since MLX targets +the Apple unified-memory GPU. + ### Convert This step is optional if you want to quantize (for QLoRA) or change the default From 99f9c952f9cfca30fa1eb2b53a6117211e0a52bd Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 21 Apr 2025 17:49:25 +0000 Subject: [PATCH 150/155] docs: note CDI spec readability for rootless Podman --- docs/containerization.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/containerization.md b/docs/containerization.md index a8f2309aaf..4bd66797ca 100644 --- a/docs/containerization.md +++ b/docs/containerization.md @@ -58,7 +58,9 @@ nvidia.com/gpu=all ``` The CDI spec should be regenerated any time the host's GPU drivers or hardware -change, otherwise `podman run` may fail to locate the devices. +change, otherwise `podman run` may fail to locate the devices. Note that +`/etc/cdi/nvidia.yaml` must be readable by the user invoking `podman run`; +rootless Podman users should verify the file's permissions after generation. ### 4. Run the container with GPU access From 3a8600019a1d86c57e96f381f6c6c91804cee3b3 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Mon, 5 May 2025 21:14:48 +0000 Subject: [PATCH 151/155] docs: add upstream license URL to generator README notice --- src/instructlab/generator/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/instructlab/generator/README.md b/src/instructlab/generator/README.md index 6780f78b8e..2904e07e14 100644 --- a/src/instructlab/generator/README.md +++ b/src/instructlab/generator/README.md @@ -1,6 +1,6 @@ ### Notice -This folder uses code from the [`stanford_alpaca`](https://github.com/tatsu-lab/stanford_alpaca) project developed by Tatsu Lab. The original code is licensed under the Apache License, Version 2.0, reproduced below for reference: +This folder uses code from the [`stanford_alpaca`](https://github.com/tatsu-lab/stanford_alpaca) project developed by Tatsu Lab. The original code is licensed under the Apache License, Version 2.0. The full text of the license is reproduced below for reference, and is also available at . ```license Apache License From ae81ca320ccc322f76676c6e4fc123c72d880388 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Tue, 6 May 2025 22:34:43 +0000 Subject: [PATCH 152/155] docs: clarify wording and improve readability of demo slides --- docs/demo-slides.md | 63 +++++++++++++++++++++++---------------------- 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/docs/demo-slides.md b/docs/demo-slides.md index b94222396f..8729b8458b 100644 --- a/docs/demo-slides.md +++ b/docs/demo-slides.md @@ -22,7 +22,7 @@ For more info, see [the CLI repo](https://github.com/instructlab/instructlab). ## Step 0: Initial setup -Create workspace and virtual environment +Create a workspace directory and a Python virtual environment. ```fish python3 -m venv workspace/venv @@ -31,14 +31,14 @@ source venv/bin/activate.fish ``` -Install CLI +Install the CLI from the stable branch. ```fish pip install git+https://github.com/instructlab/cli.git@stable ``` -Initialize workspace +Initialize the workspace. ```fish ilab init @@ -48,44 +48,46 @@ ilab init ## Step 1: Check current model behavior -Download current model +Download the current model. ```fish ilab download ``` -Start chat +Start an interactive chat session. ```fish ilab chat ``` -Trying... -> how to initialize a workspace using the CLI of InstructLab? +Try asking: + +> How do I initialize a workspace using the CLI of InstructLab? + -...but the answer is wrong +Note: the answer is incorrect at this point. --- ## Step 2: Write seed examples for SDG -Check current status of taxonomy +Check the current status of the taxonomy. ```fish ilab diff ``` -Check YAML file to add +Inspect the YAML file to add. ```fish cat ../qna.yaml ``` -Add file to taxonomy and check updated status of taxonomy +Add the file to the taxonomy and re-check its status. ```fish mkdir -p taxonomy/knowledge/instructlab/cli @@ -97,18 +99,18 @@ ilab diff ## Step 3: Generate synthetic data -Generate 25 synthetic samples +Generate 25 synthetic samples. ```fish ilab generate --num-instructions 25 ``` -- Here we use 25 for demo purpose only. -- Usually we recommend more samples (e.g. 100). +- We use 25 here for demo purposes only. +- In practice, more samples are recommended (e.g. 100). -*It takes a while, feel free to fast-forward if you are watching the recording.* +*This takes a while; feel free to fast-forward if you are watching the recording.* -Check output +Check the output. ```fish ls @@ -119,18 +121,18 @@ ls generated ## Step 4: Update model via LoRA training -Perform Q-LoRA for 100 iterations +Run Q-LoRA training for 100 iterations. ```fish ilab train --iters 100 ``` -- Here we use 100 for this particular demo. -- The actual iterations needed depends on the complexity of the data. +- We use 100 iterations for this particular demo. +- The number of iterations needed depends on the complexity of the data. -*It takes a while, feel free to fast-forward if you are watching the recording.* +*This takes a while; feel free to fast-forward if you are watching the recording.* -Check output +Check the output. ```fish ls @@ -141,7 +143,7 @@ ls instructlab-merlinite-7b-lab-mlx-q ## Step 5: Check before/after behavior -Run old/new model on test data +Run the old and new models on test data. ```fish ilab test @@ -151,14 +153,14 @@ ilab test ## Step 6: Interact with updated model -Convert updated model to GGUF +Convert the updated model to GGUF format. ```fish ilab convert --model-dir instructlab-merlinite-7b-lab-mlx-q ``` -Serve updated model and chat +Serve the updated model and start a chat session. ```fish ilab serve --model-path instructlab-merlinite-7b-lab-mlx-q-fused-pt/*-Q4_K_M.gguf & @@ -166,16 +168,17 @@ ilab chat ``` -Trying again... -> how to initialize a workspace using the CLI of InstructLab? +Try the same question again: -...and the answer is now correct! +> How do I initialize a workspace using the CLI of InstructLab? + +The answer should now be correct. --- ## Step -1: Make contribution to taxonomy -Commit change and push to fork +Commit the change and push to your fork. ```fish cd taxonomy @@ -188,8 +191,6 @@ git push --set-upstream xukai92 demo ``` -*Now, you can go creating a pull request on the taxonomy repository!* - ---> [instructlab/taxonomy](https://github.com/instructlab/taxonomy) +*Now you can open a pull request on the taxonomy repository:* [instructlab/taxonomy](https://github.com/instructlab/taxonomy). For more info, see [the CLI repo](https://github.com/instructlab/instructlab). From 943963fe446dd6910201a8174bcdaa7c9b2b0062 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Thu, 8 May 2025 16:47:30 +0000 Subject: [PATCH 153/155] Add impact and scope section to feature request template --- .github/ISSUE_TEMPLATE/feature_request.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 5076de09b2..558a043a03 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -19,5 +19,8 @@ assignees: '' **Accessibility considerations** +**Impact and scope** + + **Additional context** From 43f67f6e9397f6a0b1459eb00ebb017fa23a4187 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 9 May 2025 02:37:03 +0000 Subject: [PATCH 154/155] Add key terms and see-also sections to knowledge wiki --- tests/testdata/temp_repo/knowledge-wiki.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/tests/testdata/temp_repo/knowledge-wiki.md b/tests/testdata/temp_repo/knowledge-wiki.md index 982053dedd..c0d39b58f5 100644 --- a/tests/testdata/temp_repo/knowledge-wiki.md +++ b/tests/testdata/temp_repo/knowledge-wiki.md @@ -10,3 +10,15 @@ many controversies focus on justification. This includes questions like how to u whether it is needed at all, and whether something else besides it is needed. These controversies intensified in the latter half of the 20th century due to a series of thought experiments called Gettier cases that provoked alternative definitions. + +## Key terms + +- **Propositional knowledge**: knowledge that some statement is true (e.g., "Paris is the capital of France"). +- **Practical knowledge**: knowing how to perform a task or skill. +- **Acquaintance knowledge**: familiarity with a person, place, or thing through direct experience. + +## See also + +- Epistemology +- Gettier problem +- Justified true belief From 5eb481f27c9dbf8386363d56c504e61a36e84f10 Mon Sep 17 00:00:00 2001 From: Logan Bailey Date: Fri, 9 May 2025 13:07:03 +0000 Subject: [PATCH 155/155] Expand testing section in PR template with env and coverage notes --- .github/PULL_REQUEST_TEMPLATE.md | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index e5d851989e..83de7b94a8 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -42,13 +42,23 @@ additional context. Use "Closes #N" or "Refs #N" as appropriate. -- [ ] Existing tests pass locally +- [ ] Existing tests pass locally (``) - [ ] New tests have been added to cover the changes +- [ ] Edge cases and error paths are exercised by tests - [ ] Manual verification was performed (describe below) +### Test Environment + + + +- OS: +- Toolchain / runtime: +- Other relevant versions: + ### Reproduction Steps @@ -57,6 +67,13 @@ reviewers can reproduce. Include relevant details about your test setup. 2. 3. +### Test Coverage Notes + + + ## Risk Assessment