Skip to content

Ale007XD/provider-fallback-demo

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

3 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

provider-fallback-demo

Demonstrates the State > Model thesis on the real llm-nano-vm stack.

"What happens if your model becomes unavailable mid-task?"

In June 2026, access to two Anthropic models (Claude Fable 5 and Claude Mythos 5) was suspended worldwide in response to a government export control directive β€” with no advance warning to the companies depending on them. Closed-model access can be revoked at any time, for reasons that have nothing to do with the model's technical performance. This demo treats that as an architectural problem, not a hypothetical one.


What the demo shows

A credit application pipeline runs through a three-step FSM. At the verify_income step, the primary provider (Claude) becomes unavailable. The FSM switches to a backup provider (GPT) and completes the task.

Two failure scenarios:

Scenario Behavior
--failure-mode retry Provider degrades: 3 attempts β†’ RetryLimitExceeded β†’ switch
--failure-mode hard Provider disappears: 1 attempt β†’ ProviderUnavailable β†’ switch

Both scenarios finish the same way:

final_status: SUCCESS
provider_final: gpt

Architectural thesis

Traditional Agent:         nano-vm:

Task                       Task
  ↓                          ↓
Claude                      FSM
  ↓                          ↓
FAIL               Claude β†’ βœ— β†’ GPT β†’ βœ“
                             ↓
                          COMPLETE

The system does not bet on a provider. It bets on preserving state.

The FSM determines the path. The LLM produces a signal inside a step. The provider is an implementation detail.


Output of --both

=== Scenario: RETRY ===

S1  collect_application   βœ“  claude

S2  verify_income
  CLAUDE failed (1/3)
  CLAUDE failed (2/3)
  CLAUDE failed (3/3)

  EVENT: RetryLimitExceeded
  ACTION: switch_provider  claude β†’ gpt

S3  policy_decision       βœ“  GPT
    final_confirmation    βœ“  GPT

RECEIPT:
{
  "final_status": "SUCCESS",
  "provider_final": "gpt",
  "switch_event": "RetryLimitExceeded",
  "trace_hash": "c6f5c32c..."
}

=== Scenario: HARD ===

S1  collect_application   βœ“  claude

S2  verify_income
  EVENT: ProviderUnavailable (CLAUDE)
  ACTION: switch_provider  claude β†’ gpt

S3  policy_decision       βœ“  GPT
    final_confirmation    βœ“  GPT

RECEIPT:
{
  "final_status": "SUCCESS",
  "provider_final": "gpt",
  "switch_event": "ProviderUnavailable",
  "trace_hash": "c6f5c32c..."
}

=== COMPARISON TABLE ===

  Metric                      Retry               Hard Cutoff
  ----------------------------------------------------------------
  final_status                SUCCESS             SUCCESS
  completed_steps             6                   6
  rejected_transitions        0                   0
  switch_event                RetryLimitExceeded  ProviderUnavailable
  provider_final              gpt                 gpt
  trace_hash                  c6f5c32ce3d9...     c6f5c32ce3d9...

  Different execution trace.  Same business outcome.

  State survives. Providers don't.

Why trace_hash is identical

Both scenarios traverse the identical FSM path: set_step_s1 β†’ s1_collect β†’ set_step_s2 β†’ try_s2 β†’ check_s2_result β†’ switch_provider β†’ s2_after_switch β†’ s3_setup β†’ s3_decision β†’ approved.

The retry logic is encapsulated inside the try_s2 TOOL step β€” the FSM never sees individual attempts, only the step's final result.

trace_hash = SHA-256(Merkle(step_results)). When the FSM path matches, the hashes match. This is a property of the construction, not a coincidence: same path β†’ same state β†’ same receipt.


Implementation

Pattern: interceptable failure via TOOL

An LLM step in nano-vm that fails marks the step FAILED and stops the trace. For the FSM to branch on a provider failure, the failure is intercepted inside a TOOL:

TOOL attempt_llm_step   β†’ returns 1 (success) or 0 (failed)
CONDITION $provider_ok < 1  β†’ then: switch_provider
                              otherwise: s3_setup
TOOL do_switch_provider β†’ updates current_provider
TOOL attempt_llm_step   β†’ retries on the new provider

The FSM only ever sees successful transitions. Provider failure is a governed event, not an exception.

Files

provider_demo/
β”œβ”€β”€ receipt_demo.py   # CLI: --failure-mode retry|hard|--both
β”œβ”€β”€ programs.py       # FSM program (provider-agnostic DSL)
β”œβ”€β”€ providers.py      # MockAdapter + FailureConfig (failure injection)
└── tools.py          # attempt_llm_step, do_switch_provider, set_current_step

Known limits

  • ExecutionVM.run() is async β€” every tool that calls the LLM adapter must be async def
  • ASTEngine conditions do not support string literals as the right-hand side of a comparison (parses, always evaluates False); the working pattern is a numeric sentinel via output_key, checked as $var < 1 / $var > 0
  • Fallback chain is a fixed list (claude β†’ gpt β†’ qwen), not a scored or ranked choice
  • MockAdapter does not call a real provider API β€” responses are deterministic by design so the demo runs without API keys

What's next

  • Stage 2: Streamlit visualization β€” FSM graph + live trace + Receipt panel
  • outcome_hash: a hash over (program_name, final_status, key_outputs) only β€” invariant with respect to provider
  • Execution Equivalence: Trace_A β‰  Trace_B, but Outcome(A) = Outcome(B) β†’ a formal equivalence relation

Related projects

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages