Pentaho Kettle Tightens Database Connections, SSH Timeouts, and Dependency Chain

Pentaho Kettle shipped thirteen commits in the past week across 839 files. The work is not a headline feature drop. It is the kind of maintenance that keeps long running ETL estates stable: dependency bumps with compatibility follow ups, a centralized database factory for Connection Management Service routing, SSH timeout semantics that finally match operator intent, and a TinkerPop graph migration for lineage analyzers.

Pentaho Data Integration remains a batch oriented ETL platform with a large plugin surface. Carte slaves, repository jobs, Kafka steps, and Pentaho Metaverse lineage all share the same core runtime. That shared classpath is why a single commons configuration2 upgrade can ripple through PropertyInput steps, or why an SSH timeout of zero can silently become a 30 second cap. This week’s commits read like an operations team closing those gaps before the next release train.

Most of the diff volume sits in pom.xml and plugin level POMs. Ramaiz Mansoor landed repeated commons lang upgrades under PPP 5970 and PPP 5960, paired with a commons configuration2 bump to version 2.15.0 under PPP 6540. The ticket titles cite non vulnerable versions, which is the usual Pentaho pattern for clearing CVE flagged Apache Commons artifacts across the parent BOM.

Vitor Cardoso reverted one commons lang upgrade on June 18, then the team landed it again on June 22. Treat the June 22 state as authoritative and regression test transformations that lean on commons lang utilities.

The latest configuration2 commit also pins commons io to 2.18.0. The inline comment in pom.xml is explicit: cfg2 2.15.0 calls IOSupplier.getUnchecked(), which only exists in commons io 2.17.0 and later. Without that override, FileHandler based callers such as the PropertyInput step can throw NoSuchMethodError at runtime. That is a compatibility fix, not another CVE closure. If you customize parent POM versions in a fork, keep commons io at or above 2.17.0 whenever configuration2 is 2.15.0.

TinkerPop moved to 3.7.3 in the same window. William Seyler’s metaverse and TinkerPop refresh touched Kafka analyzer integration tests, meta inject POMs, and MetaInjectAnalyzer.java. A follow up unit test repair confirms the graph API shift was not free. Lineage features that depend on pentaho metaverse should be smoke tested after any Kettle upgrade that pulls these commits.

Denys Kondratiuk’s database factory merge is the largest behavioral addition. The new DatabaseInterfaceFactory in core becomes the single entry point for creating DatabaseInterface instances from a type string. Blank types resolve to ConnectionManagementServiceMeta, a placeholder for Connection Management Service connections that carry no local database type and bind at runtime by connection id. Any other value routes through PluginRegistry lookup by id or name.

public static DatabaseInterface create( String databaseTypeDesc ) throws KettleDatabaseException {
  if ( isConnectionManagementServiceType( databaseTypeDesc ) ) {
    return new ConnectionManagementServiceMeta();
  }
  // plugin registry resolution follows
}

That design pushes instanceof checks out of scattered business logic. The same merge adds ConnectionManagementServiceMeta, SecretsManagementException, and a reworked CmsTokenProvider that posts token requests through HttpClientManager. Roughly 700 lines of new tests cover the factory, CMS provider, and secrets exception types.

Bruno Cavaleiro’s SSH timeout fix closes a footgun that has been open since the MINA SSHD abstraction landed. Setting timeout to 0 in SSH job entries was supposed to mean infinite wait. The code treated timeout > 0 as the only finite case and silently substituted 30000 ms when the value was zero or negative.

The fix flips the guard to timeout <= 0 for infinite semantics and converts positive values with long arithmetic to avoid overflow. Three MINA SSHD 2.x call sites now use no argument await() and verify() for zero or negative timeouts instead of await(0L) and verify(0L), which poll rather than wait.

If your Carte jobs move large files over SFTP with timeout set to 0, this change removes unexplained 30 second failures on slow links. If you relied on the accidental 30 second default, behavior will change. Audit SSH and SFTP steps that use timeout 0 before upgrading.

Alan Bryant added a one line but meaningful change in RunJobServlet: clearBowlCache now runs before loadJob when Carte serves the runJob endpoint. Bowl is Kettle’s shared object cache for repository metadata. Without clearing it, repeated HTTP triggered job runs could see stale repository state. The new test asserts cache clearing happens before job load, in order.

Miguel Appleton’s reporting output classloader fix adds @ParentFirst( patterns = { ".*" } ) to PentahoReportingOutputMeta. Pentaho Reporting pulls conflicting library versions when child first classloading wins. Parent first loading for that step addresses BISERVER 15274 style failures where report generation dies on classpath clashes inside PDI.

Three items deserve attention before you roll these commits into a production Carte cluster.

First, rerun transformations that use PropertyInput, Meta Inject lineage, or Pentaho Reporting Output. The dependency and classloader changes are the highest risk surface. Second, review SFTP and SSH job entries where timeout is 0 or unset. Infinite wait is now real, which helps bulk file moves but can leave hung connections visible longer. Third, if you maintain a custom parent POM, mirror the commons io 2.18.0 pin alongside commons configuration2 2.15.0. Skipping it recreates a runtime NoSuchMethodError that security scanners will not flag.

The week was mostly plumbing. For Pentaho shops running hundreds of nightly batch jobs, that plumbing is what keeps the scheduler trustworthy.