feat: Improve inference CLI and update tests by MartinekV · Pull Request #166 · BioGeMT/agentomics-ml

MartinekV · 2026-09-06T15:47:16Z

This PR

adds CLI inference support for using any model artifacts (e.g. from a retrained model)
fixes forking bugs
- now it uses the actual forked run config, not the original run's
- environment recreation bug that potentially prevented forked runs from installing new packages
update codebase to expose better testing interfaces
establishes testing standards
revamp existing tests to meet those standards (added, removeed, and updated the whole test suite)
dientangle the test runner from agentomics-run
introduce end-to-end tests

I wasnt able to run GPU tests, so please run them on a GPU machine during the review

andrea-gariboldi · 2026-09-09T15:06:37Z

+                f"must be symbolic links, not copies. Up to first 10: {names}"
+            )
+
    CLASSIFICATION_MINI_TRAIN_SAMPLES_PER_CLASS = 100


I would keep these two class constants (lines 86 and 87) above the definition of the validate_generated_split method (like we had before this change)

Implemented

andrea-gariboldi · 2026-09-09T16:41:27Z

        agent_id=target_agent_id,
        workspace_dir=str(target_workspace_dir),
    )
+    save_config(target_config)


Is this save necessary? The config file would be saved regardless at the beginning of the (forked) run, in initialize_run (line 84 of run_agent.py). Could we leave target_config in memory for environment restoration and let initialization persist the final config?

Unrelated to your changes: I would rename the save_config method to something that communicates about saving to file (e.g. write_config_file, or save_config_to_file)

Config is loaded from the filesystem through the run - we need to update it with the new agent id, otherwise it will still have the old runs agent id. Currently this doesnt have any bugs linked to it afaik, but there could be a time where that discrepancy introduces a new bug.
Renaming was implemented.

andrea-gariboldi · 2026-09-10T10:12:12Z

        return None

-    def create_model(self, model_name: str, config: Config) -> OpenAIModel:
+    def create_model(self, model_name: str, config: Config) -> Model:


As providers can now return a generic Model type, should this method return a generic Model and not a OpenAIModel (line 109)? As well the docstring now is mentioning OpenAI sdk friendly

The Model is an abstract class. We should keep the default OpenAI model return here since its the most widely compatible one.

Makes sense. I was trying to point out the inconsistency between code and docstring, your change in the dostring is satisfactory from my end

andrea-gariboldi · 2026-09-14T14:34:06Z

Could we retain the environment-export regression scenarios covering Conda overwriting pip packages, differing Conda/pip package names, and base-environment package leakage? These tests existed because those interactions are difficult to predict. The replacement mixed-environment test doesn’t cover all of them. Rewriting them through the production export/recreation interfaces makes sense, but I’d preserve the scenarios.
In particular I am referring to:
test_graphviz_pip_conda_name_mismatch_survives_recreation,
test_graphviz_conda_only_survives_recreation,
test_pip_before_conda_order_survives_recreation,
test_export_captures_actual_version_after_conda_overwrites_pip,
test_export_excludes_phantom_packages_from_base_env

Let me know what you think.

Implemented these and added some reasonable parametrizable tests that could be easily expanded in the future if we find failing cases.
Skipped test_export_excludes_phantom_packages_from_base_env since i dont think it covers a realistic regression.

andrea-gariboldi · 2026-09-14T15:36:41Z

        default=Path(Config.BEST_ITERATION_SNAPSHOT_DIRNAME),
        help="Iteration directory relative to --agent-dir",
    )
+    parser.add_argument(


Could we put --all-iterations and --artifacts-dir in a parser.add_mutually_exclusive_group()? This would reject the combination during
parsing with a standard CLI error.

If you are applying this change, the test test_inference_requires_one_iteration_when_artifacts_are_supplied in tests/unit/test_inference_cli.py would need to expect SystemExit with code 2 during parsing instead of ValueError during execution.

Implemented

andrea-gariboldi · 2026-09-14T16:49:35Z

+        validate_docker_gpu_access(image)
+
+    container_name = f"agentomics-tests-{uuid4().hex}"
+    docker_arguments = ["--name", container_name, "--pull", "never"]


Among the arguments for the created containers, we should pass also the proxy env variables (if set).

Implemented

andrea-gariboldi · 2026-09-14T19:38:00Z

+        environment = {
+            key: value for key, value in os.environ.items()
+            if key in {
+                "PATH", "DOCKER_HOST", "DOCKER_CONTEXT", "DOCKER_CONFIG",


In here the proxy vars should be forwarded as end-to-end workflows recreate conda environments which require network access.

Implemented

andrea-gariboldi · 2026-09-15T08:41:01Z

+    parser.add_argument("--cpu-only", action="store_true", help="Disable GPU access and GPU tests")
+    arguments = parser.parse_args(argv)
+
+    image = arguments.image or build_development_image()


The development image-build path needs the proxy support from PR #164. Could we incorporate that change once merged? In here build_development_image() is called, so no separate build-proxy implementation should be needed here.

This is automatically solved by merging with main that includes PR 164 since the proxy env variables are automatically fetched from the .env file
(will rebase)

andrea-gariboldi

Good job on this. I left some minor comments.
I can confirm I run the tests on a machine with GPU, and the tests are passing.

andrea-gariboldi · 2026-09-16T10:28:23Z

Small additional note: I was able to run the tests on our server after patching mysefl the proxy env vars forwarding.

Extract env creation in tests Add a graphviz recreation test Add conda package recreation test Parametrize env recreation tests with package infos Add pip-conda-pip order env recreation test Add a pip-conda order env recreation test add pip and pip-conda packages env recreation tests

… the environment exports

andrea-gariboldi

Comments from round 1 have been addressed. I added a very small cleanup of an import.

MartinekV added 7 commits September 6, 2026 17:34

Expose runtime seams for deterministic tests

8163f0e

Fix forked run environment restoration

c443b1f

Replace the legacy tests with pytest suites

474ec0c

Add scripted end-to-end workflow tests

454f12c

Separate test execution from the agent run CLI

6c0f9b5

Support custom inference artifact directories

a4f0f84

Document the project testing standard

ea987c4

andrea-gariboldi self-requested a review September 9, 2026 09:39

andrea-gariboldi reviewed Sep 9, 2026

View reviewed changes

andrea-gariboldi self-assigned this Sep 10, 2026

andrea-gariboldi reviewed Sep 10, 2026

View reviewed changes

andrea-gariboldi reviewed Sep 14, 2026

View reviewed changes

andrea-gariboldi reviewed Sep 15, 2026

View reviewed changes

andrea-gariboldi requested changes Sep 15, 2026

View reviewed changes

MartinekV added 11 commits September 23, 2026 09:44

Merge branch 'main' into test_standards

99a77b1

Propagate proxy vars into test docker images

8ef7675

Move mini train size constants

db24ea2

Improve inference arg parsing of artifacts dir

21e0a39

Update save config fn name

b963b89

Adjust create_model docstring

d9277e7

Forward proxy vars to test containers

05e0353

Add no proxy env var passing across docker containers

1bf91a1

Fix proxy env passing to test containers

bcdd0aa

Preserve same-named but logically distinct-from-pip conda packages in…

a493d6b

… the environment exports

MartinekV and others added 5 commits September 24, 2026 09:23

Correctly remove pip-replaced conda packages from env export

be17649

Update gitignore

f09c9d0

Fix pip re-installation test

1bab623

Defer a failing env recreation test

3d3e674

Fix import line duplication

88cc3df

andrea-gariboldi approved these changes Sep 28, 2026

View reviewed changes

andrea-gariboldi merged commit dc353bb into main Sep 28, 2026
2 checks passed

agentomics-release-automation Bot mentioned this pull request Sep 28, 2026

chore: release 1.2.0 #168

Open

Conversation

MartinekV commented Sep 6, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrea-gariboldi Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrea-gariboldi Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MartinekV Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrea-gariboldi left a comment

Choose a reason for hiding this comment

Uh oh!

andrea-gariboldi commented Sep 16, 2026

Uh oh!

andrea-gariboldi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

andrea-gariboldi Sep 9, 2026 •

edited

Loading

andrea-gariboldi Sep 14, 2026 •

edited

Loading

MartinekV Sep 23, 2026 •

edited

Loading