# IQVigilant™ — the AI-Native Dynamic Engine

> IQVigilant is a pip-installable extension (`pip install iqvigilant`) for apps built on the
> [VeloIQ™ framework](https://github.com/cesarlugos1s/veloiq) (FastAPI · SQLModel /
> SQLAlchemy 2.0 · React · Ant Design). It adds two authorable-without-code capabilities:
> safe AI agents (deterministic / agentic / hybrid) and natural-language querying & charting.
> Capabilities are licensed by group via a signed JWT scoped to one installation.

This file is written for developer AI assistants (Claude, Cursor, Copilot). It gives the
**exact** wiring parameters — verified against the engine source, not the marketing snippets —
so generated integration code is correct on the first try.

## Critical accuracy notes (read before generating code)

- IQVigilant is **not a standalone SDK you drop into a bare FastAPI app.** It loads as a VeloIQ
  extension. The host app must be built on `veloiq-framework` and must *explicitly opt in*.
- Installing the wheel does **not** auto-activate it. The host enables it once via
  `veloiq extend-package iqvigilant`, which writes `[extensions].enabled` in the host's
  `veloiq.toml`. Startup and `veloiq generate` both gate on that allowlist.
- The framework discovers the package through the entry point
  `[project.entry-points."veloiq.extensions"] iqvigilant = "iqvigilant.manifest:ExtensionManifest"`.
- Dynamic Business Rules, Page Builder, Journeys, and AI-Assisted Data Import are **not** part of
  this package. Do not generate code against
  `iqvigilant.modules.business_rules`/`.pageconfig`/`.journey`/`.data_import_export`; those import
  paths only exist as deprecated re-export shims during the migration window.

## Install & activate (host app)

```bash
pip install iqvigilant
veloiq extend-package iqvigilant      # writes veloiq.toml [extensions].enabled = ["iqvigilant", ...]
veloiq generate                       # copies frontend schemas/components, regenerates routes
veloiq run                            # modules light up in menus, command center, search
```

`veloiq remove-package iqvigilant` disables it; `veloiq list-extensions` shows installed-vs-enabled.
Backend modules live under `iqvigilant.modules.*` and are loaded by the framework's three-pass
loader (models → custom_api/api routers → admin views), the same way the host's own modules load.

## Licensing (gates every capability)

Module groups (`MODULE_GROUPS` in `modules/license/license_registry.py`) — the `module_group`
JWT claim must match a key **exactly**:

| Group | Gated modules |
|---|---|
| `Natural Language` | `nlp` |
| `Exception Management` | `exception_alert`, `action`, `mechanism`, `vigilant_process` |

A JWT is verified with the bundled RSA public key, scoped to the installation UUID, with
`start_date`/`end_date` and per-group usage caps. Status resolves to `active` (full),
`grace_period` (read-only), or `blocked`. Helpers: `is_module_accessible(name)` (active or grace)
and `is_module_write_allowed(name)` (active only).

Natural Language JWT claims: `installation_id`, `module_group:"Natural Language"`,
`max_nlp_users`, `start_date`, `end_date`.
Exception Management JWT claims: `installation_id`, `module_group:"Exception Management"`,
`max_exception_alerts`, `start_date`, `end_date`.

License API (`modules/license/custom_api.py`, prefix `/api/iqvigilant/license`):
`GET /pool` (returns `caps` with real-time usage: `{natural_language: {nlp_users: [cur, max]},
exception_management: {exception_alerts: [cur, max]}}`), `GET /installation-id`, `GET /keys`,
`POST /keys`, `PATCH /keys/{id}/deactivate`.

Runtime enforcement:
- `max_exception_alerts` counted via `SELECT COUNT(*) FROM jm_exceptionalert`.
- `max_nlp_users` enforced in-memory (no DB table — tamper-proof).  A new NLP user is rejected
  (403) when the count reaches the limit.  Re-login by the same user frees their slot.

Concurrent-login prevention: in-memory single-session-per-user.  On login the middleware
hashes the JWT and stores `user_id → token_hash`.  A second login by the same user evicts
the first session.  Subsequent requests with the old token return `401 "Session superseded"`.
No database table — cannot be bypassed via DB access.

Issue keys with the JuiceMantics licensing tool (`jm_licensing` — a separate internal repository for RSA keygen + JWT signing).

- [Licensing](pricing.html)

---

## Module: nlp — natural-language querying, charting & AI agents

Database-backed NL engine plus the Agentic / Deterministic execution layer. Deterministic,
agentic and hybrid modes work together — users choose when to promote probabilistic plans into
deterministic ones. The LLM translates into predefined actions, not executes them, cutting
tokens and avoiding hallucination.

- [Safe AI Agents](learn-ai-agents.html) · [Querying & Charting](learn-analytics.html)

Entities: `NLChat`, `NLSentence` (+ junctions) using `StandardModel` (`eid` PK), `jm_`-prefixed
tables/columns (`jm_nlchat`, `jm_nlsentence`, …), exposed as `jm_nlchat` / `jm_nlsentence` resources.

LLM provider via env: `LLM_PROVIDER` ∈ `openai | watsonx | ollama | mistral`, plus
`OPENAI_API_KEY` / `WATSONX_API_KEY` / etc.

**Required spaCy models** — the SVO parser (`semantic_SVO_parser.py`) calls `spacy.load(...)`
with a locale-specific trained model, not just the base `spacy` package. Under the shipped
default `[nlp] enable_nlp_using_external_libraries = true`, that's `en_core_web_trf` (English)
and `es_dep_news_trf` (Spanish) — install both (`python -m spacy download en_core_web_trf` /
`es_dep_news_trf`) in the host's environment, or set `enable_nlp_using_external_libraries =
false` and install the smaller offline fallbacks `en_core_web_lg` / `es_core_news_lg` instead.
Missing the model for a sentence's locale makes every SVO parse fail silently (OSError caught
internally) and every answer degrade to the generic "I could not answer your request..."
fallback — with no visible error unless `[logging] log_up_to_relevance` (or
`VELOIQ_NLP_LOG_LEVEL`) is turned up, since the exception is logged at a relevance level
suppressed by production defaults.

Runtime behavior is read from `jm_config.ini` (package default → host `config/jm_config.ini` /
`jm_config.ini`, later overrides earlier). Env overrides: `VELOIQ_NLP_ANSWER_VIA_LLM`,
`VELOIQ_NLP_RENDER_LOAD_TYPE` (`eager|lazy`), `VELOIQ_NLP_LOG_LEVEL`. Flags
`use_agentic_planner_and_deterministic_executor` and `use_workflow_agent` enable the agent layer.

Agent layer (`modules/nlp/nlp_agentic_logic/`): deterministic plan executor + LRM planner + ReAct
workflow agent. Tools come from the user-defined registry `config/nl_tools/nl_tools.csv` (name +
description per row) plus generic NLP/schema tools. Plan storage (host working dir):
`deterministic_plan_registry.json` (active plans), `potential_deterministic_plans.txt` (candidates);
a packaged sample registry is the fallback until the host writes its own. UIs: Deterministic Plan
Manager, Deterministic Plan Builder, API Tools Manager, API Tool Configuration (host dropdown →
Configurations → AI Management Console). The **API Tool Configuration** page (`ApiToolConfig.tsx`) lets you
create, edit, auto-discover, test, and delete external API tools with HTTP-method-aware mutation
guards (GET executes immediately; POST/PUT/PATCH/DELETE require confirmation). Auto-Discover
uses heuristic URL analysis — no OpenAPI spec needed — to detect path params (numeric→integer,
UUID/dates/alpha→string) and query params from the URL. The Test API button shows response
status, headers, duration, and a collapsible formatted-JSON body in a modal. All text is
i18n-wrapped (`_()`) with translations in the host app's
`config/internationalization/locales/{en,es}/LC_MESSAGES/messages.po`.

The **API Tools Manager** (`ApiToolsManager.tsx`) now has two vertically stacked cards:
- **Registered API Tools** — the existing table of active tools.
- **Candidate Tools** — a browsable catalog of 15 pre-configured API tools shipped with IQVigilant
  (Frankfurter currency rates, Nager.Date public holidays, Open-Meteo weather, goQR.me QR codes,
  Nominatim geocoding, REST Countries data, Zippopotam postal codes, World Bank indicators,
  WorldTimeAPI timezones,
  plus 4 built-in Python utilities for regex parsing and datetime normalization).  Candidates are
  promoted into the active registry individually or in bulk via the `RocketOutlined` icon button —
  the same candidate→promotion pipeline used by Deterministic Plans.  Backend: `api_candidates.py`
  and `builtin_utilities.py`.

The **deterministic executor** now performs hard-dependency analysis on tool failures: steps whose
output is not required by later steps are skipped (conditional skipping); critical failures halt
execution with a polite localized message, suppressing raw stack traces from user output.

**Retry limits** for PandasAI, agent execution, and SQL queries are configurable in
`jm_config.ini` `[nlp]` (keys: `pandasai_max_retries`, `max_agent_execution_retries`,
`sql_max_retries`, `sql_retries_sleep_seconds`) and editable in the AI Management Console.
Log messages show `X of Y` progress (e.g., `pandasai_retries: 1 of 3`).

The Manager's **Deterministic Savings** tab quantifies the core promise — eliminating hallucinations
and cutting token consumption — with two locale-aware Plotly charts (by month and by plan) showing the
input/output tokens that deterministic and hybrid (agentic-planner + deterministic) executions saved.
Savings are stamped per `NLAnswer` execution from the plan's recorded agentic baseline; no LLM is re-run
to measure them. A separate **Natural Language Analytical Queries Vigilance** page (host dropdown →
Configurations → AI Management Console → AI Vigilance section) applies the same idea to the native NL engine:
whenever it answers an analytical query / chart / summary WITHOUT an LLM or agents, it credits a
configurable per-answer text-to-query token estimate (`[nlp] text_to_query_estimated_input/output_tokens`)
onto the `NLAnswer` and charts the estimated savings by month and by action type.
Show-overrides `NLChatShow` / `NLSentenceShow` stream the engine's result HTML.

**Suggested Sentences** — a "Suggest me what to ask" bulb, mounted via an antd `Popover`
(never an inline-expanding panel or a modal, so it stays usable in tight spaces like a
Dashboard cell toolbar or the tab strip) on every page a user is already on:
NL Chat, every host model's List/Show page (`list_header_button_components` /
`show_header_button_components` extension points — reusable by any extension, not
IQVigilant-specific), an Exception Alert's own Show page plus the site-wide alert
banner, and every Dashboard cell/tab (`dashboard_tab_header_components`). Regenerated
fresh on every open (no caching); accepting one or more sentences creates/adds to an
NL Chat and opens it in a new pane. Implemented in
`nlp_agentic_logic/suggested_sentences.py`; fails closed under the same conditions as
every other answering path (license/config/LLM availability).

## Module: license — IQVigilant license enforcement (always exempt)

See **Licensing** above. Endpoints under `/api/iqvigilant/license`. This module is never gated.

## Module: exception_alert — conditional alerting engine

DB-backed (`Exceptiontype`, `Exceptiondef`, `Exceptionalert` SQLModel tables with `jm_` prefix).
Alerts are generated by Exception Definitions — each `Exceptiondef` carries a `rule_query_string`
(SQL SELECT returning entity eids) and belongs to one `Exceptiontype`. When conditions match, an
`Exceptionalert` row is created and linked to the entity via the polymorphic
`alert_applies_to_object` junction. The JSON in the `/api/iqvigilant/license/pool` response maps to
`check_exception_management_caps()`.

API: standard CRUD on `jm_exceptiontype`, `jm_exceptiondef`, `jm_exceptionalert`. Licensed under
**Exception Management**.

## Module: action — mitigation actions

DB-backed (`Action` SQLModel table with `jm_` prefix). Actions record mitigation/resolution steps
applied to exception alerts. A single Action can be linked to multiple alerts via
`applied_actions` (M2M). Action → Mechanism binding: an Action can implement one or more
`Mechanism` rows via `action_implements_mechanism`.

API: standard CRUD on `jm_action`. Licensed under **Exception Management**.

## Module: mechanism — automated NL-driven remediation

DB-backed (`Mechanism` SQLModel table with `jm_` prefix). A Mechanism links to one or more
`NLSentence` rows via `mechanism_is_defined_using`. When a user applies an Action to an
Exceptionalert and that Action implements a Mechanism, the Mechanism executes its linked
NL Sentences — the entity the alert applies to is passed as context, enabling fully automated
remediation (reorder stock, reassign resource, send notification) without manual scripting.

API: standard CRUD on `jm_mechanism`. Licensed under **Exception Management**.

## Module: vigilant_process — background process execution

DB-backed (`VigilantProcess`, `VigilantProcessExec` SQLModel tables with `jm_` prefix).
Tracks and monitors long-running background processes. Execution records persist start/end
timestamps, status, and result metadata.

API: standard CRUD on `jm_vigilantprocess`, `jm_vigilantprocessexec`. Licensed under
**Exception Management**.

---

## Conventions for generated code

- New extension tables: prefix `iqvigilant_`. NLP tables: `jm_` prefix. Never import from the host's
  `app.modules.*` — the extension is self-contained.
- Use `veloiq_framework` building blocks: `TimestampedModel` / `StandardModel`, `jm_relationship`
  (never raw SQLAlchemy `relationship()` / SQLModel `Relationship`), `get_session`,
  `auth.utils.get_current_user`, `require_role`.
- Frontend schema files follow the three-file pattern: `{module}Schema.gen.ts` (regenerated),
  `{module}Schema.manual.ts` (never overwritten), `{module}Schema.ts` (merged). After editing,
  run `veloiq generate` from the host app.

## Reference

- Capabilities overview: [index](index.html)
- Capability guides: [Learn hub](learn.html)
- Source / packaging: [GitHub](https://github.com/juicemantics/iqvigilant) ·
  [PyPI](https://pypi.org/project/iqvigilant/) ·
  [VeloIQ Framework](https://github.com/cesarlugos1s/veloiq)
- Contact: [Talk to us](contact.html)