Migrating datasources

In legacy agents, the datasource was tied to a single ORM (Sequelize, Mongoose, ActiveRecord, Mongoid). The new agent introduces an explicit datasource layer that supports multiple databases and APIs in the same agent, and decouples the connection from the framework.

API cheatsheet

Legacy agent	New agent
`forest-express-sequelize`	`@forestadmin/agent` + `@forestadmin/datasource-sql`
`forest-express-sequelize` (with Sequelize models)	`@forestadmin/agent` + `@forestadmin/datasource-sequelize`
`forest-express-mongoose`	`@forestadmin/agent` + `@forestadmin/datasource-mongoose`
`forest-rails` (ActiveRecord)	`forest_admin_agent` + `forest_admin_rails` + `forest_admin_datasource_active_record` (+ toolkit, customizer)
`forest-rails` (Mongoid)	`forest_admin_agent` + `forest_admin_rails` + `forest_admin_datasource_mongoid` (+ toolkit, customizer)

Pick datasource-sql or datasource-mongo if you want Forest to introspect the database directly. Pick datasource-sequelize, datasource-mongoose, or datasource-active-record if you want to reuse your existing ORM models (recommended when your application already defines them).

Before (Node.js, forest-express-sequelize)

const Liana = require('forest-express-sequelize');
const models = require('./models');

app.use(Liana.init({
  modelsDir: __dirname + '/models',
  envSecret: process.env.FOREST_ENV_SECRET,
  authSecret: process.env.FOREST_AUTH_SECRET,
  sequelize: models.sequelize,
}));

After (Node.js, reusing Sequelize models)

import { createAgent } from '@forestadmin/agent';
import { createSequelizeDataSource } from '@forestadmin/datasource-sequelize';
import { sequelize } from './models';

const agent = createAgent({
  authSecret: process.env.FOREST_AUTH_SECRET,
  envSecret: process.env.FOREST_ENV_SECRET,
  isProduction: process.env.NODE_ENV === 'production',
});

agent.addDataSource(createSequelizeDataSource(sequelize));

agent.mountOnExpress(app).start();

After (Node.js, direct SQL connection)

If you don’t want to reuse your ORM models, connect directly to the database. Forest introspects the schema automatically.

import { createAgent } from '@forestadmin/agent';
import { createSqlDataSource } from '@forestadmin/datasource-sql';

const agent = createAgent({
  authSecret: process.env.FOREST_AUTH_SECRET,
  envSecret: process.env.FOREST_ENV_SECRET,
  isProduction: process.env.NODE_ENV === 'production',
});

agent.addDataSource(
  createSqlDataSource({
    uri: process.env.DATABASE_URL,
    sslMode: 'preferred',
  })
);

agent.mountOnExpress(app).start();

Before (Ruby, forest-rails with ActiveRecord)

# Gemfile
gem 'forest_liana'

# config/initializers/forest_liana.rb
ForestLiana.env_secret = ENV['FOREST_ENV_SECRET']
ForestLiana.auth_secret = ENV['FOREST_AUTH_SECRET']

After (Ruby, ActiveRecord)

# Gemfile
gem 'forest_admin_agent'
gem 'forest_admin_rails'
gem 'forest_admin_datasource_toolkit'
gem 'forest_admin_datasource_customizer'
gem 'forest_admin_datasource_active_record'

Run rails generate forest_admin_rails:install to scaffold the two configuration files.

# config/initializers/forest_admin_rails.rb
ForestAdminRails.configure do |config|
  config.auth_secret = ENV.fetch('FOREST_AUTH_SECRET')
  config.env_secret = ENV.fetch('FOREST_ENV_SECRET')
end

# app/lib/forest_admin_rails/create_agent.rb
module ForestAdminRails
  class CreateAgent
    def self.setup!
      database_configuration = Rails.configuration.database_configuration
      datasource = ForestAdminDatasourceActiveRecord::Datasource.new(database_configuration[Rails.env])

      @create_agent = ForestAdminAgent::Builder::AgentFactory.instance.add_datasource(datasource)
      customize
      @create_agent.build
    end

    def self.customize
      # Collection customizations (Smart Actions, computed fields, segments) go here.
    end
  end
end

Multi-datasource

The biggest payoff of migrating: the new agent can connect to multiple data sources at once.

const agent = createAgent({ /* ... */ });

// Primary database (Sequelize models reused)
agent.addDataSource(createSequelizeDataSource(sequelize));

// Analytics database (direct SQL)
agent.addDataSource(
  createSqlDataSource(process.env.ANALYTICS_DATABASE_URL),
  { name: 'analytics' }
);

// Stripe data
agent.addDataSource(
  createStripeDataSource({ secretKey: process.env.STRIPE_SECRET_KEY })
);

Cross-datasource relationships are first-class. See Relationships.

Configuration changes

Concept	Legacy	New
`envSecret`	`FOREST_ENV_SECRET`	`FOREST_ENV_SECRET` (unchanged)
`authSecret`	`FOREST_AUTH_SECRET`	`FOREST_AUTH_SECRET` (unchanged)
Models directory	`modelsDir` option	Provided implicitly via the datasource (Sequelize / ActiveRecord)
Including / excluding tables	Manual model filtering	`{ include: [...], exclude: [...] }` on `addDataSource`
Schema generation	Forest CLI / runtime	`.forestadmin-schema.json` written on agent start in development
Custom routes	Express routes	Hooks (see Hooks)

Including or excluding collections

In v1, you’d manually filter the models passed in. In v2:

agent.addDataSource(
  createSequelizeDataSource(sequelize),
  { exclude: ['internal_logs', 'session_data'] }
);

// Or only include specific collections
agent.addDataSource(
  createSequelizeDataSource(sequelize),
  { include: ['users', 'orders', 'products'] }
);

These options only hide collections: with datasource-sql, the whole schema is still read at boot. To avoid that, cache the introspection.

Match the names your layout already uses

Your layout refers to collections and fields by name, so the new agent must expose the same names as the old one. A mismatch raises no error: the old configuration just stops applying. Whether names change depends on what each agent reads:

Agent or datasource	Publishes
`forest-rails`	database column names
`forest-express-sequelize`	Sequelize attribute names, which differ from the columns with `underscored` or a `field` mapping
`datasource-sequelize`, `datasource-mongoose`, `datasource-active-record`	the model’s attribute names
`datasource-sql`, `datasource-mongo`	the database’s column or field names

So a Sequelize project that keeps datasource-sequelize usually has nothing to rename, while one moving to datasource-sql does. Restore the old names where they differ:

// Collections: a function, or an explicit map
agent.addDataSource(createSqlDataSource(process.env.DATABASE_URL), {
  rename: { user_accounts: 'userAccounts' },
});

// Fields: on the collection customizer
agent.customizeCollection('userAccounts', collection =>
  collection.renameField('first_name', 'firstName'),
);

Prove the match before you deploy

You can check names without a running environment, by diffing the new agent’s schema against the legacy one:

Capture the introspection once

import { introspect } from '@forestadmin/datasource-sql';
import { writeFileSync } from 'fs';

// Same connection options as your createSqlDataSource call, schema included.
const introspection = await introspect(process.env.DATABASE_URL);

writeFileSync('./introspection.json', JSON.stringify(introspection));

Generate the new schema without a database

Build models from that file, with no database connection:

import { createAgent } from '@forestadmin/agent';
import { buildDisconnectedSequelizeInstance } from '@forestadmin/datasource-sql';
import { createSequelizeDataSource } from '@forestadmin/datasource-sequelize';
import { readFileSync } from 'fs';

const introspection = JSON.parse(readFileSync('./introspection.json', 'utf8'));
const sequelize = await buildDisconnectedSequelizeInstance(introspection, () => {});

// The same mapping your real agent uses.
const rename = { user_accounts: 'userAccounts' };

await createAgent({
  authSecret: process.env.FOREST_AUTH_SECRET,
  envSecret: process.env.FOREST_ENV_SECRET,
  isProduction: false,
  schemaPath: './.forestadmin-schema.v2.json', // not the legacy agent's file
})
  .addDataSource(createSequelizeDataSource(sequelize), { rename })
  .generateSchemaOnly();

Use the same rename and customizations as your real agent.

Diff against the legacy schema

Compare the collection and field names in .forestadmin-schema.v2.json with the legacy .forestadmin-schema.json, adding rename and renameField entries until they match.

Once the names match, your existing layout applies to the new agent.

Validate the migration

After swapping the datasource:

Start the new agent

Run on a different port from the legacy agent (e.g. 3001).

Check schema generation

On first start in development, the agent writes a .forestadmin-schema.json file. Confirm every collection you expect is listed.

Smoke-test the API

Hit http://localhost:3001/forest, which should return Forest metadata.

Point a test environment at the new agent

Update the agent URL in your Forest test environment. Browse collections. Every collection should appear with the same fields.

Continue with customizations

Once data flows correctly, move on to migrating Smart Actions, Smart Fields, and the rest.

Common issues

A column shows up with a different name

Some legacy agents auto-converted snake_case to camelCase. The new agent preserves database column names by default. Use the rename option or the field-rename customization API to align names.

A relationship I had isn't appearing automatically

Foreign keys defined in the database are detected automatically. Relationships defined only in your ORM models (belongsTo, hasMany) are detected when you use the ORM-backed datasources (datasource-sequelize, datasource-mongoose, datasource-active-record). Relationships that exist only in code paths the agent doesn’t see won’t be detected. Declare them explicitly with addManyToOneRelation, addOneToManyRelation, etc.

The agent fails to connect to the database

Verify that DATABASE_URL is set on the agent server, that the user has read permissions, and that SSL is configured if your database requires it. For SQL datasources, check sslMode.

Next step

Migrate Smart Actions

Convert each Smart Action to the new addAction API.

Overview

JavaScript Agents

Ruby Agent

API cheatsheet

Before (Node.js, forest-express-sequelize)

After (Node.js, reusing Sequelize models)

After (Node.js, direct SQL connection)

Before (Ruby, forest-rails with ActiveRecord)

After (Ruby, ActiveRecord)

Multi-datasource

Configuration changes

Including or excluding collections

Match the names your layout already uses

Prove the match before you deploy

Validate the migration

Common issues

Next step

Migrate Smart Actions

​API cheatsheet

​Before (Node.js, forest-express-sequelize)

​After (Node.js, reusing Sequelize models)

​After (Node.js, direct SQL connection)

​Before (Ruby, forest-rails with ActiveRecord)

​After (Ruby, ActiveRecord)

​Multi-datasource

​Configuration changes

​Including or excluding collections

​Match the names your layout already uses

​Prove the match before you deploy

​Validate the migration

​Common issues

​Next step

Migrate Smart Actions

API cheatsheet

Before (Node.js, forest-express-sequelize)

After (Node.js, reusing Sequelize models)

After (Node.js, direct SQL connection)

Before (Ruby, forest-rails with ActiveRecord)

After (Ruby, ActiveRecord)

Multi-datasource

Configuration changes

Including or excluding collections

Match the names your layout already uses

Prove the match before you deploy

Validate the migration

Common issues

Next step