← MCP server and skill grades

skills

lllllllama/RigorPilot-Skills/skills

D
Type
Skill
Scanned
Nov 13, 2026
Version
fb3ccdf5aa64
Files read
126

RigorPilot-Skills is a set of AI agent skills and helper scripts for analyzing deep learning repositories, reproducing documented runs, and managing exploratory experiments. Its biggest risk is that planning an exploration run on an untrusted repository executes Python files from that repository directly in the tool's process, and symlinked READMEs or relative links can expose local files outside the repository even without running repository commands.

What it can reach

Files
Reads target repositories, creates git branches and sibling worktrees for exploration, writes evidence bundles (repro_outputs, train_outputs, explore_outputs, analysis_outputs, debug_outputs, sources, artifacts/assets), optionally writes RIGORPILOT_README.md beside a repository README, and appends persistent lesson history under ~/.rigorpilot/.
Shell commands
Executes README-documented inference, evaluation, and training commands, environment setup tools (conda, mamba, venv, pip), git commands, and process/GPU utilities (nvidia-smi, rocm-smi, taskkill).
Network
Sends outbound HTTP/HTTPS requests to arXiv, DOI, GitHub APIs, arbitrary URLs scraped from repository files during exploration lookup, and the Anthropic Messages API (or a configured endpoint) when run_agent.py is invoked.
Secrets
Reads ANTHROPIC_API_KEY (or a configured credential_env variable) in run_agent.py for model API calls, and checks optional RESEARCH_LOOKUP_*_API_KEY environment variables.
System
Samples local CPU, memory, disk usage, process CPU/RSS via /proc or Win32 APIs, and device-level GPU telemetry.

Findings

2 high3 medium2 low
  1. HighTarget repository Python modules are executed in-process during exploratory planning

    skills/ai-research-explore/scripts/passes/execution_feasibility.py:33

    When orchestrate_explore.py runs without --run-selected-variants (its default planning-only mode) or even after safety checkpoints block variant execution, it still runs import_probe_check and constructor_probe_check on candidate .py files in the target repository. Those checks add the repository root to sys.path and call spec.loader.exec_module(module) on up to four Python files (skipping only exact filenames train.py, eval.py, main.py, and __main__.py). Any top-level code in files such as model.py, backbone.py, or loss.py executes immediately inside the orchestrator process with no timeout and full access to the user's environment variables and API keys.

    Fix: Replace spec.loader.exec_module in import_probe_check and constructor_probe_check with static AST inspection (ast.parse) when --run-selected-variants is not enabled, and run any opted-in runtime smoke probes in an isolated subprocess with a timeout and filtered environment.

  2. HighSymlinked README.md escapes the repository to disclose local files and write outside the repo

    skills/repo-intake-and-plan/scripts/scan_repo.py:62

    scan_repo.py (and its bundled copy under skills/ai-research-reproduction/_bundled/) resolves README.md with readme.resolve() without checking that the resolved target remains inside the repository root. When orchestrate_repro.py runs on an untrusted git repository containing a symlinked README.md (even without --run-selected), write_annotated_readme copies the external target file verbatim into repro_outputs/ANNOTATED_README.md where the AI agent reads it. In addition, because scan_repo.py already resolved the path, write_source_adjacent_readme's readme_path.is_symlink() check evaluates to False, allowing --source-adjacent-readme to write RIGORPILOT_README.md into the external target's parent directory.

    Fix: Reject any README that is a symlink or whose resolved path is not relative to root.resolve() in both scan_repo.py copies and in write_annotated_readme before reading bytes or computing source-adjacent output paths.

  3. MediumPath traversal via Markdown links in README during reproduction planning

    skills/ai-research-reproduction/scripts/orchestrate_repro.py:1185

    When a repository's README contains no runnable commands, delegate_to_docs follows relative Markdown links ending in .md—even during --plan-only runs—without verifying that the resolved path stays inside repo_path. A crafted link such as [Quick Start](../../../other-project/README.md) lets an untrusted repository read external .md files on the machine, expose extracted command lines in command_candidates and status.json, and inject external commands into goal selection.

    Fix: Verify that target.is_relative_to(repo_path.resolve()) and reject symlink escapes before passing linked documentation files to extract_commands.py.

  4. MediumExploration lookup automatically fetches every URL in repository files without SSRF or size guards

    skills/ai-research-explore/scripts/lookup/providers/base.py:68

    By default (enable_repo_local_extraction=True), run_lookup_pass scans up to 120 code, config, and documentation files in the target repository, extracts every http:// and https:// URL, and fetches each one through resolve_url_record and http_get. Unlike agent_provider.py, http_get allows plaintext HTTP, follows redirects, permits localhost, 127.0.0.1, link-local metadata (169.254.169.254), and private network IPs, and calls response.read() with no size limit. Running orchestrate_explore.py on an untrusted repository can trigger unintended GET requests to local or internal services, leak the user's IP to external servers, exhaust memory, and write remote HTML title/meta text into agent-read artifacts.

    Fix: Do not automatically fetch generic URLs extracted from repository files unless explicitly enabled, block loopback/link-local/private IP ranges and cross-host redirects, require HTTPS, and cap response.read() to a small byte limit.

  5. MediumResearch campaign mode executes evaluation_source.command even when execution is disabled

    skills/ai-research-explore/scripts/orchestrate_explore.py:2066

    orchestrate_explore.py defaults execution_policy.run_selected_variants to False and reports status as planned when no variants are run, and SKILL.md states execution helpers are used only when real execution evidence is required. However, whenever --research-campaign-json is supplied, orchestrate_explore.py unconditionally executes campaign['evaluation_source']['command'] inside the target repository during run_baseline_evaluation before checking any execution opt-in or human checkpoint. If an agent builds a campaign file from an untrusted repository's README expecting a planning-only run, the command executes immediately.

    Fix: Gate run_baseline_evaluation behind an explicit execution flag (such as --run-baseline or --run-selected-variants) and keep default campaign runs strictly read-only during planning.

  6. LowPath traversal when loading cached source records from sources/index.json

    skills/ai-research-explore/scripts/lookup/cache_store.py:123

    When output_dir is inside an untrusted repository (making sources_dir point to <repo>/sources), store_records reads <repo>/sources/index.json and resolves any artifact_path starting with 'sources/' using Path(existing_rel).relative_to('sources') without checking path containment. A pre-populated sources/index.json using '../' segments can cause the tool to parse local JSON files outside sources_dir and merge their fields into explore_outputs/status.json.

    Fix: Resolve existing_path and verify existing_path.is_relative_to(sources_dir.resolve()) with no symlinks before reading cached JSON records.

  7. LowUntrusted README command text is persisted into the shared ~/.rigorpilot lesson store

    skills/ai-research-reproduction/scripts/orchestrate_repro.py:70

    When orchestrate_repro.py runs with --run-selected (enabled by default for lessons via RIGORPILOT_LESSONS=1), maybe_record_lesson appends failure and resolution summaries containing untrusted README command text (including unfound executable names even when launch fails before any repo code runs) to ~/.rigorpilot/lessons.jsonl. When distilled via lessons_store.py summarize into ~/.rigorpilot/PERSONAL_RIGOR.md, crafted text from one repository can persist into standing prompt context read by future agent sessions on unrelated repositories.

    Fix: Record structured error codes rather than raw README command strings in lesson summaries, and escape or strip instruction-like prose before rendering PERSONAL_RIGOR.md.