49 lines
2.4 KiB
Markdown
49 lines
2.4 KiB
Markdown
|
# Bazel Dependency Vendoring (Reference Cache)
|
||
|
|
|
||
|
|
This project uses a "Reference Cache" strategy for Bazel dependencies to allow AI agents and developers to inspect external source code without impacting build stability or concurrency.
|
||
|
|
|
||
|
|
## The Strategy
|
||
|
|
|
||
|
|
1. **Shared Location:** Dependencies are vendored into a shared directory **outside** the workspace (e.g., `../yesod_vendored_deps`).
|
||
|
|
2. **Symlink:** The local `vendor` directory is a symlink to that shared location.
|
||
|
|
3. **Read-Only:** This directory is intended for **reading and searching** source code.
|
||
|
|
4. **No Build Impact:** Standard Bazel builds (`bazel build //...`) do **NOT** use this directory (they use the standard internal Bazel cache). This prevents concurrency issues when multiple git worktrees run builds simultaneously.
|
||
|
|
|
||
|
|
## Setup for New Worktrees
|
||
|
|
|
||
|
|
If you are setting up a new git worktree, simply create the symlink:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Assuming the shared vendor dir is at ../yesod_vendored_deps
|
||
|
|
ln -s ../yesod_vendored_deps vendor
|
||
|
|
```
|
||
|
|
|
||
|
|
## Updating the Vendor Cache
|
||
|
|
|
||
|
|
If `MODULE.bazel` changes, you may want to update the shared cache. Because `bazel vendor` can fail on complex cross-platform targets (like NVIDIA drivers or private OCI images), we use an **Allowlist Strategy**: we only vendor the "safe" repositories (Go, Python, Starlark) that are useful for reading.
|
||
|
|
|
||
|
|
### How to Re-Vendor
|
||
|
|
|
||
|
|
1. **Generate the Clean Repo List:**
|
||
|
|
Run the following command to generate a list of all fetchable repositories, filtering out known problem areas (NVIDIA, OCI, etc.):
|
||
|
|
|
||
|
|
```bash
|
||
|
|
bazel mod dump_repo_mapping "" | python3 -c "import json, sys; m = json.load(sys.stdin); print('\n'.join(m.values()))" | grep -vE "nvidia|coder_dev_base_image|distroless|noble|gevent|triton|audioop|mappings_test_external_repo|^$" > clean_repo_list.txt
|
||
|
|
```
|
||
|
|
|
||
|
|
2. **Run Bazel Vendor:**
|
||
|
|
Execute the vendor command targeting *only* the clean list. We perform this step in a temporary directory or the main worktree, then move the result to the shared location.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# 1. Run vendor command
|
||
|
|
cat clean_repo_list.txt | xargs -I {} echo "--repo=@@{}" | xargs bazel vendor --vendor_dir=vendor
|
||
|
|
|
||
|
|
# 2. Move to shared location (if updating)
|
||
|
|
# BE CAREFUL: this replaces the existing cache
|
||
|
|
rm -rf ../yesod_vendored_deps
|
||
|
|
mv vendor ../yesod_vendored_deps
|
||
|
|
|
||
|
|
# 3. Restore local symlink
|
||
|
|
ln -s ../yesod_vendored_deps vendor
|
||
|
|
```
|