Plugins

Overview  

Every connector in Amfelt is a plugin. A project declares which implementations it uses and at what version, and the agent loads them at run time. Selecting plugins per role is what makes the platform composable: a new source does not require a new platform.

Roles  

Plugins are grouped by the role they play in a Project :

A single connector can appear in more than one role. Many databases are both a plausible source and a plausible target, and the plugin declares which it supports rather than the platform assuming.

Capabilities Are Declared, Not Assumed  

A plugin declares what it can do (discover, extract, or load) à la carte. Nothing requires a connector to implement all three, and Amfelt will not ask it to do something it has not claimed.

This is why the Connector Catalog can grow unevenly and still be honest: a source that supports discovery and extraction but not loading says so, and the project file cannot reference it as a target by mistake.

Plugins exchange data as Apache Arrow record batches, so data keeps one format on its way from a source plugin, through the agent, to a target plugin. At either end a plugin still works a record at a time: a source plugin reads records and packs them into batches, and a target plugin unpacks each batch and writes its records.

Where Plugins Run  

By default, a plugin runs as a local subprocess on the agent host and serves the agent over gRPC on a loopback port (127.0.0.1), which other hosts cannot reach. A plugin configured with the grpc registry instead runs at the address you set, which can be another machine, and the agent sends it the resolved connection, password included.

That matters for the security model: a local plugin adds no listener that other hosts can reach. It does connect out to the system it serves, and the agent downloads it through the control plane. The Security Model covers the boundary.

Versioning  

You can pin a plugin to a version in the project file, alongside the rest of your configuration. A plugin left unpinned, as the control plane writes it, runs the newest version installed on the agent host, or the latest from the plugin catalog when none is installed. For a pinned plugin, the project’s Git history shows the version each commit asked for; for a local plugin, a run’s events name the version the agent started.

Next  

• Projects : where plugins are declared
• Pipelines : what the plugins are used to build
• Integrations : the connectors available today