feat: full ISPConfig remote API coverage + re-runnable generator (v0.2)
v0.1 shipped ~15 hand-audited methods across sites/dns/mail/databases/clients. That's enough for daily ops but every new Tort Host / cWHO feature has been hitting the wall at the edge of that coverage. This extends the SDK to wrap every method the panel exposes — 312 of them as of Rackham 2026-04-22, verified against the live list_functions() introspection call with only one name-mismatch (``__construct``, a PHP lifecycle artifact, not a real API method). The hand-audited helpers stay where they are. Every module now has two clearly-delimited sections: an auto-generated block at the top (emitted by tools/gen_methods.py from tools/method_inventory.json), and a HAND-EDIT ONLY BELOW block at the bottom that survives regeneration. Name collisions between auto and hand always resolve in favor of the hand version — the generator emits a ``# skipped foo: hand-audited helper takes precedence`` comment in the auto block for traceability. Pipeline: - tools/extract_inventory.py reads remote.*.inc.php + remoting.inc.php, pulls docblocks + param defaults, dumps one JSON record per method. Regex is balanced-paren aware so ``$params = array()`` defaults don't truncate the signature at the wrong close-paren (that footgun hid three methods from the first run — sites_aps_available_packages_list, sites_aps_instance_delete, openvz_vm_add_from_template). - tools/method_inventory.json is the committed inventory — future ISPConfig upgrades diff against this file to see scope at a glance. - tools/gen_methods.py groups by method-name prefix onto the module classes listed in the README table, emits a 1:1 Python wrapper per method with the original PHP filename + line number in the docstring, and ensures ``from typing import Any`` is present in preexisting modules before emitting ``Any`` type annotations. New submodules (all auto-generated, wired into ISPConfigClient.__init__): admin, aps, backups, cron, domains, ftp, misc, monitor, openvz, server, shell, webdav. Existing modules (sites, dns, mail, databases, clients) got their auto block filled in and their hand-audited helpers preserved. Escape hatches on the top-level client: - raw_call(method, *args) routes an arbitrary method name through the same session-management + retry + fault-mapping pipeline the typed wrappers use. Fix for "panel shipped a new method, SDK hasn't caught up" — callers don't have to reach back into _soap. - list_functions() wraps get_function_list() for panel introspection. Fault mapping widened: ``no_client_found`` and "no user account" messages now map to NotFoundError instead of FaultError, matching the existing ``no_domain_found`` convention. Older code that caught raw FaultError there will still work (NotFoundError extends ISPConfigError) but callers can now catch the specific type. Testing: - tests/test_unit.py — 12 existing pure-unit tests pass unchanged. - tests/test_smoke.py — extended from 4 read-only calls to 21. One probe per new auto-generated module plus raw_call and list_functions smoke tests. Methods gated behind admin permission skip gracefully with a documented reason (kayos is a reseller, not admin): monitor_jobqueue_count, sites_cron_get, sites_ftp_user_get, openvz_get_free_ip, quota_get_by_user. Results against Rackham 2026-04-22: 28 passed, 5 skipped (all documented admin-only), 0 failed. - ISPCONFIG_TEST_VERIFY_SSL=0 env-var knob added to conftest for panels with self-signed or mismatched certs. Version bump 0.1.0 -> 0.2.0. README restructured into Hand-audited / Auto-generated / Escape hatch / Footguns sections with a regeneration recipe for future ISPConfig upgrades. Ruff per-file SLF001 ignore extended to every submodule (submodules are all authorized callers of the client's private ``_call`` dispatcher by design). mypy strict passes; ruff check passes; ruff format applied across src / tools / tests.
This commit is contained in:
parent
9438b4e751
commit
44ce76cb44
28 changed files with 14173 additions and 184 deletions
186
README.md
186
README.md
|
|
@ -4,8 +4,10 @@ Python SDK for the ISPConfig remote SOAP API. Internal tooling for the Sulkta
|
|||
Coop — wraps the panel's SOAP surface so we stop writing throw-away PHP
|
||||
scripts every time we need to touch a site, zone, or mailbox.
|
||||
|
||||
The SDK covers the methods we actually use today (sites, DNS, mail, databases,
|
||||
clients). More methods can be added as needed.
|
||||
v0.2 covers the **full remote API** — every method exposed by ISPConfig's
|
||||
`remote/index.php`, 312 of them as of Rackham 2026-04-22. The hand-audited
|
||||
helpers (stable names, param-order fixes, convenience wrappers) sit on top
|
||||
of auto-generated wrappers that mirror the PHP surface 1:1.
|
||||
|
||||
## Why no zeep?
|
||||
|
||||
|
|
@ -45,9 +47,35 @@ managed internally — callers don't touch them.
|
|||
|
||||
Set `verify_ssl=False` for dev boxes with self-signed certs. Default is `True`.
|
||||
|
||||
## Modules at a glance
|
||||
## Changelog
|
||||
|
||||
### `sites`
|
||||
### 0.2.0 — 2026-04-22
|
||||
|
||||
Full-coverage extension. Every remote method ISPConfig exposes is now wrapped
|
||||
in one of the submodules below. Hand-audited helpers from v0.1 (footgun
|
||||
fixes, convenience wrappers) are preserved beneath a delimiter and survive
|
||||
regeneration. New submodules: `admin`, `aps`, `backups`, `cron`, `domains`,
|
||||
`ftp`, `misc`, `monitor`, `openvz`, `server`, `shell`, `webdav`.
|
||||
|
||||
- Code generator at `tools/gen_methods.py` driven by `tools/method_inventory.json`.
|
||||
- One-command refresh for future ISPConfig upgrades (see below).
|
||||
- `ISPConfigClient.raw_call(method, *args)` escape hatch for methods not yet
|
||||
in the inventory.
|
||||
- `ISPConfigClient.list_functions()` introspects the panel's exposed method list.
|
||||
- Live smoke test extended from 4 to 21 read-only calls covering every module.
|
||||
|
||||
### 0.1.0 — 2026-04-21
|
||||
|
||||
Initial release. Hand-audited coverage of sites / dns / mail / databases / clients.
|
||||
|
||||
## Modules
|
||||
|
||||
### Hand-audited (stable API, verified in prod)
|
||||
|
||||
Names normalized for ergonomics; footgun fixes and retries baked in. These
|
||||
are the wrappers you should prefer.
|
||||
|
||||
#### `sites`
|
||||
|
||||
```python
|
||||
c.sites.web_domain_get(156)
|
||||
|
|
@ -56,38 +84,94 @@ c.sites.enable_php(156, mode="php-fpm", server_php_id=2, pm="ondemand")
|
|||
c.sites.enable_letsencrypt(156)
|
||||
```
|
||||
|
||||
### `dns`
|
||||
#### `dns`
|
||||
|
||||
```python
|
||||
zone_id = c.dns.zone_get_id("example.com.") # note the trailing dot
|
||||
zone_id = c.dns.zone_get_id("example.com") # trailing dot is stripped either way
|
||||
zone = c.dns.zone_get(zone_id)
|
||||
records = c.dns.rr_get_all_by_zone(zone_id)
|
||||
rr_id = c.dns.a_add(0, {"zone": zone_id, "name": "www", "data": "1.2.3.4", "ttl": 3600, "active": "Y"})
|
||||
```
|
||||
|
||||
### `mail`
|
||||
#### `mail`
|
||||
|
||||
```python
|
||||
md = c.mail.domain_get_by_domain("example.com")
|
||||
users = c.mail.user_get({"email": "%@example.com"})
|
||||
users = c.mail.user_get({"email": "%@example.com"}) # always a list
|
||||
new_id = c.mail.create_mailbox(client_id=5, domain="example.com",
|
||||
local_part="info", password="x", quota_mb=2048)
|
||||
```
|
||||
|
||||
### `databases`
|
||||
#### `databases`
|
||||
|
||||
```python
|
||||
db = c.databases.get(42)
|
||||
c.databases.user_update(client_id=5, primary_id=42, params={"database_password": "x"})
|
||||
```
|
||||
|
||||
### `clients`
|
||||
#### `clients`
|
||||
|
||||
```python
|
||||
cli = c.clients.get_by_username("jacob")
|
||||
groupid = c.clients.get_groupid(cli["client_id"])
|
||||
ids = c.clients.get_all()
|
||||
```
|
||||
|
||||
### Auto-generated (full surface, v0.2)
|
||||
|
||||
Wrappers mirror the PHP method names 1:1. Param shapes come from PHPDoc where
|
||||
available and default to `Any` otherwise. Verified to wire up against Rackham
|
||||
2026-04-22 but not yet battle-tested in production use — file issues if you
|
||||
hit one.
|
||||
|
||||
| Module | Class | Methods |
|
||||
|--------------|----------------------|---------|
|
||||
| `admin` | `AdminModule` | 10 |
|
||||
| `aps` | `ApsModule` | 10 |
|
||||
| `backups` | `BackupsModule` | 2 |
|
||||
| `clients` | `ClientsModule` | 19 |
|
||||
| `cron` | `CronModule` | 4 |
|
||||
| `databases` | `DatabasesModule` | 9 |
|
||||
| `dns` | `DnsModule` | 87 |
|
||||
| `domains` | `DomainsModule` | 5 |
|
||||
| `ftp` | `FtpModule` | 5 |
|
||||
| `mail` | `MailModule` | 82 |
|
||||
| `misc` | `MiscModule` | 4 |
|
||||
| `monitor` | `MonitorModule` | 1 |
|
||||
| `openvz` | `OpenvzModule` | 22 |
|
||||
| `server` | `ServerModule` | 12 |
|
||||
| `shell` | `ShellModule` | 4 |
|
||||
| `sites` | `SitesModule` | 29 |
|
||||
| `webdav` | `WebdavModule` | 4 |
|
||||
|
||||
Every auto-generated method carries a docstring with the original PHP
|
||||
filename + line number and an `AUTO-GENERATED — param shapes may need
|
||||
verification` warning. Methods that exist in both the auto and hand-audited
|
||||
blocks are skipped in the auto block — the hand version wins.
|
||||
|
||||
### Escape hatch
|
||||
|
||||
For methods not yet in the inventory (e.g. on a newer ISPConfig version than
|
||||
we've regenerated against):
|
||||
|
||||
```python
|
||||
result = c.raw_call("some_new_method", arg1, arg2)
|
||||
```
|
||||
|
||||
`raw_call` routes through the same session-management + fault-mapping
|
||||
pipeline as typed methods, so auth/retry still works. To see what the panel
|
||||
exposes:
|
||||
|
||||
```python
|
||||
funcs = c.list_functions()
|
||||
```
|
||||
|
||||
### Not covered
|
||||
|
||||
- `__construct` — PHP constructor, not a real API method.
|
||||
- Anything gated by ISPConfig plugins we don't have installed — probing via
|
||||
`raw_call` will return a faultstring like "Method not found".
|
||||
|
||||
## Footguns (captured here so nobody has to rediscover them)
|
||||
|
||||
- **`sites_web_domain_update`'s second arg is `client_id`, not `primary_id`.**
|
||||
|
|
@ -112,9 +196,9 @@ groupid = c.clients.get_groupid(cli["client_id"])
|
|||
- **`mail_user_get` with a filter dict returns inconsistent shapes.** If the
|
||||
filter matches multiple rows you get an array; exactly-one match returns a
|
||||
bare map. Our `mail.user_get(filter_dict)` always normalizes to a list.
|
||||
- **`no_domain_found` fault.** Both `dns_zone_get_id` and (in some paths)
|
||||
`mail_domain_get_by_domain` return a SOAP fault with `faultcode=no_domain_found`
|
||||
when the record is missing. Mapped to `NotFoundError`.
|
||||
- **`no_domain_found` / `no_client_found` faults.** These are typed as
|
||||
`NotFoundError`; `client_get_by_username("nope")` raises it. Older code
|
||||
may have caught raw `FaultError` — v0.2 reclassifies both.
|
||||
- **`dns_a_add` type-column bug.** On some ISPConfig versions (<= ~3.2.11)
|
||||
`dns_a_add` inserts the `dns_rr` row without setting the `type` column,
|
||||
so BIND never emits the record. `DnsModule.a_add(..., fix_type_bug=True)`
|
||||
|
|
@ -130,6 +214,17 @@ groupid = c.clients.get_groupid(cli["client_id"])
|
|||
- **Filter dicts on `mail_user_get`.** Pass an int to get one row; pass a
|
||||
dict like `{"email": "%@example.com"}` to get a list. The SOAP method is
|
||||
overloaded and untyped on the wire.
|
||||
- **PHP method signatures with `array()` defaults.** ISPConfig's extract
|
||||
regex used to stop at the first `)` in `$params = array()`, missing three
|
||||
methods on 3.2.x (`sites_aps_available_packages_list`,
|
||||
`sites_aps_instance_delete`, `openvz_vm_add_from_template`). Fixed in
|
||||
`tools/extract_inventory.py` with balanced-paren matching — worth
|
||||
double-checking on future panel upgrades.
|
||||
- **Known admin-only methods.** Reseller logins fault with "permission denied"
|
||||
on a non-admin user. These are skipped in the smoke tests:
|
||||
`monitor_jobqueue_count`, `sites_cron_get`, `sites_ftp_user_get`,
|
||||
`openvz_*`, `quota_get_by_user`, `client_templates_get_all`. Use an admin
|
||||
login if you need them.
|
||||
|
||||
## Errors
|
||||
|
||||
|
|
@ -147,7 +242,7 @@ ISPConfigError
|
|||
## Tests
|
||||
|
||||
```bash
|
||||
pytest # unit tests only, no network
|
||||
pytest # unit tests only, no network (12 tests)
|
||||
```
|
||||
|
||||
To run the live smoke test against a real panel:
|
||||
|
|
@ -156,11 +251,47 @@ To run the live smoke test against a real panel:
|
|||
export ISPCONFIG_TEST_URL="https://panel.example.com:8080/remote/index.php"
|
||||
export ISPCONFIG_TEST_USER="kayos"
|
||||
export ISPCONFIG_TEST_PASS="..."
|
||||
export ISPCONFIG_TEST_VERIFY_SSL=0 # for self-signed certs
|
||||
pytest tests/test_smoke.py
|
||||
```
|
||||
|
||||
The smoke test is read-only — no `_add` / `_update` / `_delete` calls. Safe
|
||||
against production.
|
||||
21 read-only calls covering every auto-generated module + the originals. No
|
||||
`_add` / `_update` / `_delete` calls — safe against production. Methods that
|
||||
require admin privileges skip gracefully with a documented reason.
|
||||
|
||||
## Regenerating for newer ISPConfig versions
|
||||
|
||||
When ISPConfig ships a new version on Rackham (or another panel), resync:
|
||||
|
||||
```bash
|
||||
# 1. Pull fresh PHP sources from the panel (sudo required; ask Cobb for creds):
|
||||
mkdir -p /tmp/ispconfig-php-src
|
||||
ssh rackham "sudo tar -cz -C /usr/local/ispconfig/interface/lib/classes/remote.d ." \
|
||||
| tar -xz -C /tmp/ispconfig-php-src/
|
||||
ssh rackham "sudo cat /usr/local/ispconfig/interface/lib/classes/remoting.inc.php" \
|
||||
> /tmp/ispconfig-php-src/remoting.inc.php
|
||||
|
||||
# 2. Re-extract the method inventory:
|
||||
python3 tools/extract_inventory.py /tmp/ispconfig-php-src tools/method_inventory.json
|
||||
|
||||
# 3. Regenerate wrappers:
|
||||
python3 tools/gen_methods.py
|
||||
|
||||
# 4. Clean up formatting + sanity check:
|
||||
ruff format src/ tools/
|
||||
ruff check src/ tools/
|
||||
mypy src/ispconfig
|
||||
pytest
|
||||
|
||||
# 5. Review and commit:
|
||||
git diff --stat
|
||||
git diff tools/method_inventory.json # new methods jump out here
|
||||
```
|
||||
|
||||
Hand-edits below the `HAND-EDIT ONLY BELOW` marker in each module survive
|
||||
regeneration. Method-name collisions between auto and hand are resolved in
|
||||
favor of the hand version; the generator emits a `# skipped ...` comment
|
||||
in the auto block for traceability.
|
||||
|
||||
## Development
|
||||
|
||||
|
|
@ -171,29 +302,6 @@ mypy src/ispconfig
|
|||
pytest
|
||||
```
|
||||
|
||||
## Not yet covered
|
||||
|
||||
The remote API surface is huge. These are intentionally left out of v0.1 —
|
||||
add as needed:
|
||||
|
||||
- `sites_web_aliasdomain_*`, `sites_web_subdomain_*`,
|
||||
`sites_web_vhost_{subdomain,aliasdomain}_*`
|
||||
- `sites_ftp_user_*`, `sites_shell_user_*`, `sites_webdav_user_*`
|
||||
- `sites_cron_*`
|
||||
- `sites_web_domain_backup`, `sites_web_domain_backup_list`,
|
||||
`mail_user_backup`, `mail_user_backup_list`
|
||||
- `dns_{aaaa,ns,srv,ptr,tlsa,ds,caa,sshfp,dname,loc,hinfo,naptr,rp,alias}_*`
|
||||
- `dns_slave_*`, `dns_zone_set_dnssec`, `dns_zone_get_by_user`, `dns_templatezone_*`
|
||||
- `mail_alias_*`, `mail_aliasdomain_*`, `mail_catchall_*`, `mail_filter_*`,
|
||||
`mail_fetchmail_*`, `mail_mailinglist_*`, `mail_policy_*`,
|
||||
`mail_relay_{domain,recipient}_*`, `mail_transport_*`,
|
||||
`mail_{whitelist,blacklist}_*`, `mail_spamfilter_*`, `mail_user_filter_*`
|
||||
- `client_add`, `client_update`, `client_delete`, `client_change_password`,
|
||||
`client_template_additional_*`, `client_templates_get_all`,
|
||||
`client_login_get`
|
||||
- `server_get`, `server_get_all`, `admin.*`, `monitor.*`, `aps.*`,
|
||||
`openvz.*`, `domains.*`
|
||||
|
||||
## License
|
||||
|
||||
MIT — see `LICENSE`.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue