# Add business logic Source: https://docs.forest.app/get-started/add-business-logic Create actions and computed fields to go beyond CRUD Forest's real power comes from the business logic you add on top of your data. This step covers the two most common building blocks: actions and computed fields. ## Create your first action Actions are custom operations your team can trigger from the UI, refunding an order, sending an email, flagging a record for review. You define them in your back-end code, so they can call external APIs, update multiple tables, or trigger any side effect you need. Add an action to a collection in your back-end: ```javascript theme={null} agent.customizeCollection('orders', collection => collection.addAction('Mark as reviewed', { scope: 'Single', execute: async (context, resultBuilder) => { await context.collection.update(context.filter, { status: 'reviewed' }); return resultBuilder.success('Order marked as reviewed'); }, }), ); ``` Restart your back-end and the action appears in the collection's **Actions** menu. Actions can have forms with dynamic fields, approval workflows, and granular permissions. See [Code-based actions](/product/process/actions/custom-actions/overview) for the full reference. ## Create a computed field Computed fields let you display values that don't exist directly in your database, a full name from first and last name, a status derived from multiple columns, a formatted price. Add a computed field the same way: ```javascript theme={null} agent.customizeCollection('customers', collection => collection.addField('fullName', { columnType: 'String', dependencies: ['firstName', 'lastName'], getValues: records => records.map(r => `${r.firstName} ${r.lastName}`), }), ); ``` Restart your back-end and the field appears in your back-office like any other field. Computed fields can also be made writable, filterable, and sortable. See [Fields](/product/process/fields/computed). ## What's next With actions and computed fields in place, you're ready to build higher-level operational tools: workspaces, workflows, and inboxes. Build workspaces, workflows, and inboxes for your team # Architecture Source: https://docs.forest.app/get-started/connect/architectures/self-hosted Deploy the Forest back-end in your infrastructure while Forest hosts the UI - the most popular deployment option The self-hosted architecture gives you full control over your data and code by hosting the Forest back-end in your own infrastructure. This is the **most popular deployment option** for production applications. ## Architecture overview Self-Hosted Architecture Diagram ### How it works 1. **Your users** access the Forest UI hosted at `app.forestadmin.com` 2. **Forest UI** makes API calls to your back-end 3. **Your back-end** runs in your infrastructure (AWS, GCP, Heroku, etc.) 4. **Your back-end** queries your database directly 5. **All data** stays within your infrastructure **Key benefit**: This architecture follows our privacy by design motto. Your data never leaves your infrastructure. ## Components | Component | Hosted by | Location | Responsibilities | | ----------------- | --------- | --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Your Back-end** | You | Your infrastructure | • Runs in Node.js or Ruby
• Direct access to your database
• Executes business logic (actions, computed fields, hooks)
• Handles authentication and authorization
• Exposes REST API for Forest UI | | **Forest UI** | Forest | `app.forestadmin.com` | • Back-office interface for your users
• Communicates with your back-end via HTTPS
• Receives only schema metadata (no data)
• Managed and updated by Forest | | **Your Database** | You | Your infrastructure | • PostgreSQL, MySQL, MongoDB, etc.
• Only accessible to your back-end
• No direct connection from Forest
• Your data never leaves your infrastructure | ## Back-end deployment You can deploy the Forest back-end in two ways, depending on your application architecture: ### In-app deployment With in-app deployment, the Forest back-end becomes part of your existing application. It runs in the same Node.js or Ruby process, sharing the same database connection pool and resources. When you deploy your application, the back-end deploys with it as a single unit. This approach is particularly valuable when you want to leverage your existing ORM models. Whether you're using Sequelize, Mongoose, or ActiveRecord, Forest can read directly from your model definitions, no need to redefine your schema. You can also reuse your existing business logic, sharing code, services, and utilities between your application and Forest without duplication. In-app Deployment Architecture ### Standalone deployment Standalone deployment runs the Forest back-end as a completely separate service. It operates in its own process or container with dedicated resources and its own database connection pool. Your application and the back-office are deployed and scaled independently. This architecture provides a cleaner separation of concerns. Your application code stays focused on your business logic, while Forest responsibilities are isolated in their own service. The two services communicate through well-defined boundaries, making your overall system architecture simpler to understand and maintain. This is particularly beneficial as your team grows, developers working on the application don't need to think about back-office concerns, and vice versa. **Note**: Standalone deployment is currently available for Node.js only. If you need standalone deployment with Ruby, please [contact our sales team](https://www.forestadmin.com/contact). Standalone Deployment Architecture **Recommendation**: We recommend **standalone deployment** for cleaner architecture and better separation of concerns. Use **in-app** if you need to reuse your ORM models or existing business logic. ## Privacy by design The self-hosted architecture is built with privacy as a core principle. Here's how data flows when a user accesses Forest: When a user opens Forest, the browser loads UI and config from Forest's public endpoints, but all data requests go directly to your Forest back-end behind your firewall, so your data never reaches Forest servers **The key principle:** Forest's servers (`app.forestadmin.com`, `api.forestadmin.com`) only send configuration and UI code to your browser. All data requests go directly from your browser to your back-end behind your firewall, and your data never leaves your infrastructure. **Looking for a different setup?** Forest also offers **on-premise** deployments where even the Forest UI runs in your own infrastructure. [Contact us](https://www.forestadmin.com/contact) to find the right fit for your organization. # ActiveRecord datasource Source: https://docs.forest.app/get-started/connect/data-sources/active-record Connect Forest to your ActiveRecord models with support for relationships and Live Query The ActiveRecord datasource enables importing collections from all model classes extending `ActiveRecord::Base`. It automatically respects your ActiveRecord relationships and configurations. ActiveRecord datasource is only available for Ruby. For Node.js, check the [Sequelize datasource](/get-started/connect/data-sources/sequelize) or [SQL datasource](/get-started/connect/data-sources/sql). ## Basic usage ```ruby theme={null} require 'forest_admin_datasource_active_record' def self.setup! ForestAdminDatasourceActiveRecord::Datasource.new( { 'adapter' => ENV['DB_ADAPTER'], 'host' => ENV['DB_HOST'], 'username' => ENV['DB_USERNAME'], 'password' => ENV['DB_PASSWORD'], 'database' => ENV['DB_DATABASE'], } ) end ``` ## Features The ActiveRecord datasource automatically preserves your ORM configuration: * **ActiveRecord relationships** - Relationships will be respected * **Live Query support** - Users with proper permissions can create Live Query components for charts, analytics, and segments using SQL ## Live Query support Enable SQL-based reporting by setting a connection name identifier when creating the datasource: ```ruby theme={null} ForestAdminDatasourceActiveRecord::Datasource.new( database_config, live_query_connections: 'main_database' ) ``` This allows authorized users to create Live Query charts, analytics, and segments that execute custom SQL directly against your database. Live Queries execute raw SQL. Ensure proper access controls and review queries before deploying to production. ## Multi-database configuration For applications using multiple databases, provide a hash mapping display names to Rails connection identifiers: ```ruby theme={null} ForestAdminDatasourceActiveRecord::Datasource.new( database_config, live_query_connections: { 'main_database' => 'primary', 'replica_database' => 'primary_replica' } ) ``` The configuration keys display in Forest's UI, while the values reference connection names from your `config/database.yml` file. ## Polymorphic relationships Forest supports polymorphic relationships in ActiveRecord by enabling a specific configuration option on the datasource instance. ### Configuration To activate polymorphic relationship support, set the second parameter to `true` when instantiating the datasource: ```ruby theme={null} def self.setup! database_configuration = Rails.configuration.database_configuration # Enable polymorphic relations with true parameter datasource = ForestAdminDatasourceActiveRecord::Datasource.new( database_configuration[Rails.env], true ) @create_agent = ForestAdminAgent::Builder::AgentFactory.instance.add_datasource(datasource) customize @create_agent.build end ``` ### Tips The `id` and `type` fields of polymorphic relationships remain visible in the interface. Hide them through back-end customization or frontend configuration if desired. ### Limitations #### Collections * **Removal**: Cannot delete collections that serve as polymorphic relationship targets; attempting this raises: "Cannot remove because it's a potential target of polymorphic relation" * **Renaming**: Cannot rename collections involved in polymorphic relationships; error states: "Cannot rename collection because it's a target of a polymorphic relation" #### Fields * **Removal**: Cannot delete fields (`foreign_key_type`, `foreign_key_id`) used in polymorphic relationships * **Renaming**: Cannot rename fields involved in these relationships * **Computed Fields**: Cannot use polymorphic relationships as dependencies for computed fields due to unpredictable target collections #### Search Extended search functionality skips polymorphic relationships to prevent excessive resource consumption, generating debug logs about this behavior. Implement custom search behavior via documentation. ## Source code This connector is open source. Browse the code or contribute on GitHub: [`forest_admin_datasource_active_record`](https://github.com/ForestAdmin/agent-ruby/tree/main/packages/forest_admin_datasource_active_record). # Airtable Source: https://docs.forest.app/get-started/connect/data-sources/airtable Connect Airtable bases to Forest Only available for Node.js. The Airtable datasource connects your Airtable bases to Forest, allowing you to browse, search, and manage your Airtable data directly from your back-office. ## Installation ```bash theme={null} npm install @forestadmin-experimental/datasource-airtable ``` ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createAirtableDataSource } from '@forestadmin-experimental/datasource-airtable'; const agent = createAgent(options); agent.addDataSource( createAirtableDataSource({ apiKey: process.env.AIRTABLE_API_KEY, baseId: process.env.AIRTABLE_BASE_ID }) ); ``` ## Source code [github.com/ForestAdmin/forestadmin-experimental, datasource-airtable](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-airtable) ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin-experimental/datasource-airtable`](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-airtable). # CosmosDB Source: https://docs.forest.app/get-started/connect/data-sources/cosmosdb Connect Azure CosmosDB to Forest Only available for Node.js. The CosmosDB datasource connects your Azure CosmosDB instance to Forest, allowing you to browse, search, and manage your CosmosDB data directly from your back-office. ## Installation ```bash theme={null} npm install @forestadmin-experimental/datasource-cosmos ``` ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createCosmosDataSource } from '@forestadmin-experimental/datasource-cosmos'; const agent = createAgent(options); agent.addDataSource( createCosmosDataSource({ endpoint: process.env.COSMOS_ENDPOINT, key: process.env.COSMOS_KEY, databaseId: process.env.COSMOS_DATABASE_ID }) ); ``` ## Source code [github.com/ForestAdmin/forestadmin-experimental, datasource-cosmos](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-cosmos) ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin-experimental/datasource-cosmos`](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-cosmos). # Overview Source: https://docs.forest.app/get-started/connect/data-sources/custom-datasources/overview Build a custom datasource for any system without an existing connector, using the translation or replication strategy When Forest has no built-in connector for your system (a proprietary database, a legacy service, a third-party REST API), you can build your own datasource. There are two strategies, and the right one depends on what your source can do and how fresh the data needs to be. ## Translation vs. Replication | | [Translation](/get-started/connect/data-sources/custom-datasources/translation) | [Replication](/get-started/connect/data-sources/custom-datasources/replication) | | --------------------- | ------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | **How it works** | Translates Forest's query interface into your API's query language and calls the source live on every request | Maintains a local copy (replica) of the source data in a cache controlled by the back-end, and queries that copy | | **Data freshness** | Always live: every request hits the source | Eventually consistent: refreshed on a schedule, by polling, or via push/webhooks | | **Query translation** | You implement it (filters, sort, pagination, aggregation) | None: filtering, sorting, search, and charts work out of the box against the cache | | **Effort** | High: a full-featured translation layer often exceeds 1000 lines | Lower: you mostly implement how data is fetched and (optionally) written back | | **Best for** | Sources that already support rich querying and where data must always be live | Slow, rate-limited, or query-poor APIs where a cached copy is acceptable | | **Availability** | Node.js and Ruby | Node.js only | ## Choosing a strategy * Reach for **Translation** when your source is a capable database or API that can filter, sort, and paginate efficiently, and you need every read to reflect the latest state. * Reach for **Replication** when the source API is slow, rate-limited, or hard to query, and a periodically synced copy is good enough. It is the simpler path and gives you Forest's full feature set with no query translation. **Don't see the datasource you need?** [Contact us](https://www.forestadmin.com/contact) to request a connector, or share your own through the [Forest experimental repository](https://github.com/ForestAdmin/forestadmin-experimental). # Replication Source: https://docs.forest.app/get-started/connect/data-sources/custom-datasources/replication Build custom datasources that maintain a local cache of external data for better performance Replication datasources are only available for Node.js. The replication strategy maintains a copy of target API data in an internal cache controlled by your back-end, rather than querying the API in real-time. Minimal replica datasource architecture ## Overview ### Key advantages * **No query translation**: No query translation logic required * **Performant**: Eliminates synchronous network calls to the target API * **Feature-complete**: Charts, filtering, and search work out of the box * **Flexible**: Implement custom logic for fetching target API data * **Robust**: Recover bad states by reconstructing the replica from scratch ### Minimal implementation ```javascript Node.js theme={null} const { createReplicaDataSource } = require('@forestadmin/datasource-replica'); const axios = require('axios'); const myCustomDataSource = createReplicaDataSource({ pullDumpHandler: async () => { const url = 'https://jsonplaceholder.typicode.com'; const collections = ['posts', 'comments', 'albums', 'photos', 'users']; const entries = []; for (const collection of collections) { const response = await axios.get(`${url}/${collection}`); entries.push(...response.data.map(record => ({ collection, record }))); } return { more: false, entries }; }, }); agent.addDatasource(myCustomDataSource); ``` This basic implementation fetches all records at startup but doesn't update them afterward. ### Known limitations & solutions | Limitation | Solution | | ---------------------------------------------------- | ---------------------------------- | | Full data dump required at each startup | Implement persistent cache | | Empty collections and foreign keys not auto-detected | Provide explicit schema definition | | Data never updates after initial import | Implement update handlers | | Read-only data | Implement write handlers | | Nested fields and arrays in API responses | Use record flattener utility | ## Persistent cache The Forest Node.js back-end uses a SQL database as its underlying cache mechanism. By default, an in-memory SQLite database is used. ### Limitations of in-memory cache The default in-memory approach presents two main challenges: 1. **Extended startup time**: The back-end must re-fetch all data from the target API on each restart 2. **High memory consumption**: All data remains in memory, which becomes problematic for large datasets ### When to use persistent cache Depending on which API you are targeting, it may be absolutely fine to use an in-memory cache for smaller datasets. However, larger systems like CRMs or databases containing millions of records benefit significantly from persistent storage. ### Cache initialization Forest will automatically detect when the schema of the tables in the caching database does not match the schema of the target API. When mismatches occur, tables and indexes are dropped, recreated, and repopulated from the target API. ### Configuration options * **`cacheInto`**: Accepts a connection string or configuration object for the SQL connector * **`cacheNamespace`**: Prefixes table names, useful for sharing databases or running multiple replicas **Important:** No locking mechanism currently exists for concurrent writes when multiple back-end instances share the same cache configuration. ### SQLite file example ```javascript Node.js theme={null} const myCustomDataSource = createReplicaDataSource({ cacheInto: 'sqlite:/tmp/my-cache.db', pullDumpHandler: async () => { return { more: false, entries: [] }; }, }); ``` ### PostgreSQL example ```javascript Node.js theme={null} const myCustomDataSource = createReplicaDataSource({ cacheInto: { uri: 'postgres://xxxx:[email protected]/neondb', sslMode: 'verify', }, cacheNamespace: 'my-custom-data-source', pullDumpHandler: async () => { return { more: false, entries: [] }; }, }); ``` ## Updating the replica Real-world scenarios require keeping the Forest back-end to display up-to-date data. ### Three update methods Use these approaches independently or combine them: 1. **Scheduled rebuilds** - Refetch all records periodically 2. **Change polling** - Uses Forest events to detect modifications 3. **Change pushing** - Leverages target API events via webhooks The target API feeds a replica cache held by the Forest back-end via three update methods (scheduled rebuild, change polling, and change pushing), and Forest queries the replica ## Scheduled rebuilds Scheduled rebuilds represent the simplest approach for updating replica data by fetching all records from a target API at regular intervals. This method works with any API but is less efficient for large datasets since it requires fetching all records regardless of changes. ### Configuration options **`pullDumpOnRestart`**: When set to `true`, data fetches on each back-end startup. This is always enabled for default in-memory cache implementations. **`pullDumpOnSchedule`**: Accepts cron-like schedule patterns for periodic updates. For example: `['0 0 0 * * *', '0 30 18 * * *']` triggers daily at midnight and 6:30 PM. ### Schedule syntax The system uses the croner NPM package for schedule parsing with this format: ``` ┌─ second (0-59) │ ┌─ minute (0-59) │ │ ┌─ hour (0-23) │ │ │ ┌─ day of month (1-31) │ │ │ │ ┌─ month (1-12) │ │ │ │ │ ┌─ day of week (0-6) * * * * * * ``` Common examples: * `* * * * * *` - Every second * `0 * * * * *` - Every minute * `0 0 9 * * 1` - Mondays at 9am ### Handler implementation The `pullDumpHandler` returns entries for import and supports pagination. The request object provides `previousDumpState` (for change detection), `cache` access, and `reasons` (startup/schedule triggers). The response object specifies entries to import, pagination via `more` flag, and state persistence through `nextDumpState` and `nextDeltaState` fields. **Key advantage:** Old data remains available to users until new data processing completes, preventing service disruption. ## Change polling Change polling is a strategy for updating replica data sources by fetching only records that have changed, rather than pulling all data from the target API on each update. ### When to poll for changes Four triggering events are available: 1. **pullDeltaOnRestart**: Handler executes when the back-end restarts 2. **pullDeltaOnSchedule**: Handler runs on a cron-like schedule (same syntax as pullDumpOnSchedule) 3. **pullDeltaOnBeforeAccess**: Handler executes before each datasource access; GUI blocks until completion 4. **pullDeltaOnAfterWrite**: Handler executes after each write operation; GUI blocks until completion **Optional delay feature:** `pullDeltaOnBeforeAccessDelay` (milliseconds) groups multiple requests sent during the delay period, reducing calls to your target API. Set to 0 to disable. ### Handler implementation Implement a `pullDeltaHandler` function that receives a request object containing: * `previousDeltaState`: Persisted state from previous calls * `affectedCollections`: Collections being accessed or written to * `cache`: Interface for reading cached data * `reasons`: Array explaining why the handler was invoked The handler should return a response object with: * `more`: Boolean indicating if additional changes exist (triggers immediate re-call) * `nextDeltaState`: State persisted for subsequent handler invocations * `newOrUpdatedEntries`: Records created or modified since last call * `deletedEntries`: Records removed since last call ## Push & webhooks The push strategy keeps replicas up-to-date when APIs expose change-following capabilities through webhooks, WebSockets, long polling, or similar mechanisms. ### Handler programming Unlike the pull strategy, developers are responsible for setting up subscriptions to the target API. The back-end calls your handler during startup to establish these subscriptions, and you send changes to the back-end for replica updates. ### Request object structure The request provides: * `getPreviousDeltaState()`: Fetches delta state asynchronously, useful when mixing push and pull strategies * `cache`: Interface for reading from the cache ### onChange payload structure The payload includes: * `nextDeltaState` (optional): Updated delta state for recovery on back-end restart * `newOrUpdatedEntries`: Array of created/updated records with collection and record data * `deletedEntries`: Array of deleted records (full record not required) ### Example: CouchDB change feed Using the nano library to subscribe to CouchDB's changes stream: ```javascript Node.js theme={null} const { createReplicaDataSource } = require('@forestadmin/datasource-replica'); const nano = require('nano'); const myCustomDataSource = createReplicaDataSource({ pushDeltaHandler: async (request, onChanges) => { const stream = nano.db.changesAsStream('books', { include_docs: true, since: await request.getPreviousDeltaState(), }); stream.on('data', change => { onChanges({ nextDeltaState: change.seq, newOrUpdatedEntries: !change.deleted ? [{ collection: 'books', record: { _id: change.id, ...change.doc } }] : [], deletedEntries: change.deleted ? [{ collection: 'books', record: { _id: change.id } }] : [], }); }); }, }); ``` ### Example: webhook implementation Using Express to receive webhooks on a separate port: ```javascript Node.js theme={null} const { createReplicaDataSource } = require('@forestadmin/datasource-replica'); const myCustomDataSource = createReplicaDataSource({ pushDeltaHandler: async (request, onChanges) => { const app = express(); app.use(express.json()); app.post('/webhooks/on-book-:type(created|change|deleted)', (req, res) => { onChanges({ newOrUpdatedEntries: req.params.type === 'created' || req.params.type === 'change' ? [{ collection: 'book', record: req.body }] : [], deletedEntries: req.params.type === 'deleted' ? [{ collection: 'book', record: { id: req.body.id } }] : [], }); res.status(204).send(); }); app.listen(3000); }, }); ``` ## Schema & references ### Schema auto-discovery When no explicit schema is provided, the back-end attempts to auto-discover structure from imported data. However, this approach has limitations: * Empty collections cannot be imported * Performance overhead from sampling data * Primary keys must be named `id` * Composite primary keys unsupported * Foreign keys aren't automatically detected ### Providing a schema Supply a schema via the `createReplicaDataSource` function to avoid auto-discovery limitations. The schema can be static or dynamically generated through Promises or async functions. ### Schema syntax **Collection definition** includes: * `name`: Collection identifier * `fields`: Object containing field definitions, supporting nested objects and arrays **Field definition** properties (type required): * Type options: Boolean, Integer, Number, String, Date, Dateonly, Timeonly, Binary, Enum, Json, Point, Uuid * `defaultValue`: Initial value for new records * `enumValues`: Possible values for Enum types * `isPrimaryKey`: Marks primary key fields * `isReadOnly`: Read-only designation * `unique`: Uniqueness constraint * `validation`: Array of validation rules * `reference`: Defines foreign key relationships with target collection details ### Handling complex data **Flatten mode** addresses limitations with nested structures and arrays. Options include `auto` or `manual` modes, similar to Mongoose driver configuration. When enabled, flatten mode: * Automatically transforms nested records * Creates virtual collections for arrays * Uses `@@@` as field separator in flattened output * Generates synthetic IDs and foreign keys for relationships **Important:** Original records in handlers remain unflattened; transformation occurs during cache import only. ## Write handlers ### Implementation requirements Three optional handlers can be implemented: `createRecordHandler`, `updateRecordHandler`, and `deleteRecordHandler`. Omit any handler for operations not needed. The `createRecordHandler` function uniquely supports return values, which proves useful when the target API auto-generates record IDs. ### Code example ```javascript Node.js theme={null} const axios = require('axios'); const { createReplicaDataSource } = require('@forestadmin/datasource-replica'); const url = 'https://jsonplaceholder.typicode.com'; const myCustomDataSource = createReplicaDataSource({ // Record synchronization implementation... createRecordHandler: async (collectionName, record) => { const response = await axios.post(`${url}/${collectionName}`, record); return response.data; }, updateRecordHandler: async (collectionName, record) => { await axios.put(`${url}/${collectionName}/${record.id}`, record); }, deleteRecordHandler: async (collectionName, record) => { await axios.delete(`${url}/${collectionName}/${record.id}`); }, }); ``` ### Key takeaways * All three write handlers remain optional * Create handlers can return newly generated IDs from the API * Update and delete handlers perform remote operations without returning values * The handlers abstract the communication layer between Forest and external APIs Want to share your custom datasource with the community? Check out the [Forest experimental repository](https://github.com/ForestAdmin/forestadmin-experimental) to contribute. # Translation Source: https://docs.forest.app/get-started/connect/data-sources/custom-datasources/translation Build your own datasource by translating Forest queries into your API's query language Translation datasources require advanced knowledge of Forest's query interface. The translation strategy is an advanced approach for creating your own datasources that involves translating Forest's query interface into the target API's query language. ## Overview A full-featured query translation module typically exceeds 1000 lines of code. This approach suits full-featured databases and requires deep understanding of Forest's internals. Translation datasource capabilities diagram ### Key steps Implementing this strategy requires completing three main phases: 1. **Structure declaration** - Define the data structure 2. **Capabilities declaration** - Specify API capabilities 3. **Translation layer implementation** - Code the actual query translation ### Minimal example ```javascript Node.js theme={null} class MyCollection extends BaseCollection { constructor(dataSource) { super('myCollection', dataSource); // Add fields with type, filtering, and sorting capabilities } async list(caller, filter, projection) { // Translate Forest query to API format const params = QueryGenerator.generateListQueryString(filter, projection); const response = axios.get('https://my-api/my-collection', { params }); return response.body.items; } } ``` ```ruby Ruby theme={null} class MyCollection < ForestAdminDatasourceToolkit::Collection def initialize(datasource) super(datasource, 'MyCollection') # Add fields with type, filtering, and sorting capabilities end def list(caller, filter, projection) # Translate Forest query to API format params = QueryGenerator.generate_list_query_string(filter, projection) response = HTTParty.get('https://my-api/my-collection', query: params) response.parsed_response['items'] end end ``` ## Structure declaration ### Columns Define fields with types, validation, and default values: ```javascript Node.js theme={null} const { BaseCollection } = require('@forestadmin/datasource-toolkit'); class MovieCollection extends BaseCollection { constructor() { // [...] this.addField('id', { type: 'Column', columnType: 'Number', isPrimaryKey: true, }); this.addField('title', { type: 'Column', columnType: 'String', validation: [{ operator: 'Present' }], }); this.addField('mpa_rating', { type: 'Column', columnType: 'Enum', enumValues: ['G', 'PG', 'PG-13', 'R', 'NC-17'], defaultValue: 'G', }); this.addField('stars', { type: 'Column', columnType: [{ firstName: 'String', lastName: 'String' }], }); } } ``` ```ruby Ruby theme={null} class MovieCollection < ForestAdminDatasourceToolkit::Collection include ForestAdminDatasourceToolkit::Schema def initialize(datasource) super(datasource, 'Movie') add_field('id', ColumnSchema.new( column_type: 'Number', is_primary_key: true )) add_field('title', ColumnSchema.new( column_type: 'String', filter_operators: [Operators::PRESENT] )) add_field('mpa_rating', ColumnSchema.new( column_type: 'Enum', enum_values: %w[G PG PG-13 R NC-17], default_value: 'G' )) add_field('stars', ColumnSchema.new( column_type: [ { 'firstName' => 'String', 'lastName' => 'String' } ] )) end end ``` ### Typing The typing system for columns is the same as the one used when declaring fields in the back-end customization step. ### Validation Forest permits declaring validation rules on primitive-type fields. These rules validate records during creation/updating in the back-office interface. The validation API mirrors the condition tree structure but excludes a "field" entry. Example validation clause: ```json theme={null} { "aggregator": "and", "conditions": [ { "operator": "present" }, { "operator": "like", "value": "found%" }, { "operator": "today" } ] } ``` ### Relationships **Important:** Only intra-datasource relationships belong at the collection level. For inter-datasource relationships, use jointures during customization. Data sources using the query translation strategy require careful implementation for relationships. ```javascript Node.js theme={null} const { BaseCollection } = require('@forestadmin/datasource-toolkit'); class MovieCollection extends BaseCollection { constructor() { // [...] this.addField('director', { type: 'ManyToOne', foreignCollection: 'people', foreignKey: 'directorId', foreignKeyTarget: 'id', }); this.addField('actors', { type: 'ManyToMany', foreignCollection: 'people', throughCollection: 'actorsOnMovies', originKey: 'movieId', originKeyTarget: 'id', foreignKey: 'actorId', foreignKeyTarget: 'id', }); } } ``` ```ruby Ruby theme={null} class MovieCollection < ForestAdminDatasourceToolkit::Collection def initialize(datasource) super(datasource, 'Movie') add_field('director', ManyToOneSchema.new( foreign_key: 'director_id', foreign_key_target: 'id', foreign_collection: 'People' )) add_field('actors', ManyToManySchema.new( origin_key: 'movie_id', origin_key_target: 'id', foreign_key: 'actor_id', foreign_key_target: 'id', foreign_collection: 'People', through_collection: 'ActorsOnMovies' )) end end ``` ## Capabilities declaration Data source implementers don't need to translate every possible query type. Forest ensures only supported query features are available by having collections declare capabilities on construction. ### Required features All datasources must support: * Listing records * `And` nodes in condition trees * `Or` nodes in condition trees * `Equal` operator on primary keys * Paging (`skip`, `limit`) **Note:** Translating the `Or` node is a strong constraint, as many backends will not allow it: providing a working implementation may require making multiple queries and recombining the results. ### Optional features (opt-in) | Unlocked feature | Required capabilities | | ---------------------------------- | ----------------------------------------------------- | | Pagination page count display | Count | | Charts | All field support in Aggregation | | Relations | `In` on primary/foreign keys | | Select all for actions/delete | `In` and `NotIn` on primary key | | Frontend filters, scopes, segments | Per-field operator support | | Operator emulation | `In` on primary keys | | Search emulation | `Contains` on strings; `Equal` on numbers/UUIDs/enums | ### UI filter requirements by field type To unlock GUI filtering: * **Boolean:** `Equal`, `NotEqual`, `Present`, `Blank` * **Date:** All date operators * **Enum:** `Equal`, `NotEqual`, `Present`, `Blank`, `In` * **Number:** `Equal`, `NotEqual`, `Present`, `Blank`, `In`, `GreaterThan`, `LessThan` * **String:** `Equal`, `NotEqual`, `Present`, `Blank`, `In`, `StartsWith`, `EndsWith`, `Contains`, `NotContains` * **UUID:** `Equal`, `NotEqual`, `Present`, `Blank` ### Collection-level capabilities #### Count Enables pagination widget to display total page count. Requires implementing the `aggregate` method: ```javascript Node.js theme={null} class MyCollection extends BaseCollection { constructor() { this.enableCount(); } } ``` ```ruby Ruby theme={null} class MyCollection < ForestAdminDatasourceToolkit::Collection def initialize # [...] enable_count end end ``` #### Search Allows custom search implementation instead of default condition tree approach. Useful for full-text search (ElasticSearch, etc.): ```javascript Node.js theme={null} class MyCollection extends BaseCollection { constructor() { this.enableSearch(); } } ``` ```ruby Ruby theme={null} class MyCollection < ForestAdminDatasourceToolkit::Collection def initialize # [...] enable_search end end ``` #### Segments Define segments at datasource level when condition trees are insufficient or segments are shared across projects: ```javascript Node.js theme={null} class MyCollection extends BaseCollection { constructor() { this.addSegments(['Active records', 'Deleted records']); // All filter-accepting methods MUST handle segment fields } } ``` ```ruby Ruby theme={null} class MyCollection < ForestAdminDatasourceToolkit::Collection def initialize # [...] add_segments(['Active records', 'Deleted records']) # All filter-accepting methods MUST handle segment fields end end ``` ### Field-level capabilities #### Write support Mark fields as read-only: ```javascript Node.js theme={null} this.addField('id', { isReadOnly: true, }); ``` ```ruby Ruby theme={null} add_field('id', { is_read_only: true }) ``` #### Filtering operators Declare supported operators per field: ```javascript Node.js theme={null} this.addField('id', { filterOperators: new Set([ 'Equal', // additional operators ]), }); ``` ```ruby Ruby theme={null} add_field('id', { filter_operators: ['Equal'] }) ``` #### Sort support Flag sortable fields: ```javascript Node.js theme={null} this.addField('id', { isSortable: true, }); ``` ```ruby Ruby theme={null} add_field('id', { is_sortable: true }) ``` ## Read implementation ### Emulation strategy Emulation enables rapid development by allowing features to be tested in Node.js before optimization. This approach trades performance for faster iteration. ### Basic list implementation ```javascript Node.js theme={null} const { BaseCollection } = require('@forestadmin/datasource-toolkit'); const axios = require('axios'); class MyCollection extends BaseCollection { async list(caller, filter, projection) { // Fetch all records const response = await axios.get('https://my-api/my-collection'); const result = response.data.items; // Apply in-process emulation if (filter.conditionTree) result = filter.conditionTree.apply(result, this, caller.timezone); if (filter.sort) result = filter.sort.apply(result); if (filter.page) result = filter.page.apply(result); return projection.apply(result); } } ``` ```ruby Ruby theme={null} class MyCollection < ForestAdminDatasourceToolkit::Collection def list(caller, filter, projection) # Fetch all records on all requests (inefficient approach) response = HTTParty.get('https://my-api/my-collection') result = response.parsed_response['items'] # Apply filtering, sorting, pagination sequentially result = filter.condition_tree.apply(result, self, caller.timezone) if filter.condition_tree result = filter.sort.apply(result) if filter.sort result = filter.page.apply(result) if filter.page # Handle unsupported operations raise 'Unsupported feature' if filter.segment || filter.search projection.apply(result) end end ``` ### Aggregate method The `aggregate` method handles both record counting and chart data generation: ```javascript Node.js theme={null} async aggregate(caller, filter, aggregation, limit) { const records = await this.list(caller, filter, aggregation.projection); return aggregation.apply(records, caller.timezone, limit); } ``` ```ruby Ruby theme={null} def aggregate(caller, filter, aggregation, limit = nil) records = list(caller, filter, aggregation.projection) aggregation.apply(records, caller.timezone, limit) end ``` ### Optimization: count queries Handle count operations separately if your API supports efficient counting: ```javascript Node.js theme={null} async aggregate(caller, filter, aggregation, limit) { if (aggregation.operation === 'Count' && aggregation.groups.length === 0) { return [{ value: await this.count(caller, filter) }]; } // Handle general case } ``` ```ruby Ruby theme={null} def aggregate(caller, filter, aggregation, limit = nil) # Optimize count-only queries if aggregation.operation == 'Count' && aggregation.groups.empty? && !aggregation.field return [{ 'value' => count(caller, filter) }] end # Handle general aggregation case end ``` ## Write implementation Making your records editable is achieved by implementing the `create`, `update` and `delete` methods. **Important:** The three write methods accept filter parameters, but unlike the `list` method, pagination support is unnecessary. ```javascript Node.js theme={null} const { BaseCollection } = require('@forestadmin/datasource-toolkit'); const axios = require('axios'); // client for the target API /** Naive implementation of create, update and delete on a REST API */ class MyCollection extends BaseCollection { constructor() { this.addField('id', { /* ... */ isReadOnly: true }); this.addField('title', { /* ... */ isReadOnly: false }); } async create(caller, records) { const promises = records.map(async record => { const response = await axios.post('https://my-api/my-collection', record); return response.data; }); return Promise.all(promises); // Must return newly created records } async update(caller, filter, patch) { const recordIds = await this.list(caller, filter, ['id']); // Retrieve ids const promises = recordIds.map(async ({ id }) => { await axios.patch(`https://my-api/my-collection/${id}`, patch); }); await Promise.all(promises); } async delete(caller, filter) { const recordIds = await this.list(caller, filter, ['id']); // Retrieve ids const promises = recordIds.map(async ({ id }) => { await axios.delete(`https://my-api/my-collection/${id}`); }); await Promise.all(promises); } } ``` ```ruby Ruby theme={null} class MyCollection < ForestAdminDatasourceToolkit::Collection include ForestAdminDatasourceToolkit::Schema def initialize(datasource) super(datasource, 'MyCollection') add_field('id', ColumnSchema.new(is_read_only: true)) add_field('title', ColumnSchema.new(is_read_only: false)) end def create(caller, records) records.map do |record| response = HTTParty.post('https://my-api/my-collection', body: record.to_json) response.parsed_response end end def update(caller, filter, patch) record_ids = list(caller, filter, ForestAdminDatasourceToolkit::Components::Query::Projection.new(['id'])) record_ids.each do |record| HTTParty.patch("https://my-api/my-collection/#{record['id']}", body: patch.to_json) end end def delete(caller, filter) record_ids = list(caller, filter, ForestAdminDatasourceToolkit::Components::Query::Projection.new(['id'])) record_ids.each do |record| HTTParty.delete("https://my-api/my-collection/#{record['id']}") end end end ``` ### Method details * **create()**: Must return the newly created records with all fields populated * **update()**: Receives filter and patch object; updates matching records * **delete()**: Receives filter; deletes all matching records ## Intra-datasource relationships When building your own datasources using the translation strategy, collections must handle intra-datasource relationships that are declared in their structure. ### Relationship types and requirements **Automatic handling:** * `one-to-many` relationships * `many-to-many` relationships For these types, Forest will automatically call the destination collection with a valid filter, requiring no additional implementation work. **Manual implementation required:** * `many-to-one` relationships * `one-to-one` relationships These require developers to make all fields from the target collection available on the source collection (under a prefix). ### Handling prefixed fields When a `many-to-one` relationship exists, the collection must accept references using dot notation throughout its operations. #### Structure declaration example ```javascript Node.js theme={null} class MovieCollection extends BaseCollection { constructor() { super('movies', null); this.addField('director', { type: 'ManyToOne', foreignCollection: 'people', foreignKey: 'directorId', foreignKeyTarget: 'id', }); } } ``` ```ruby Ruby theme={null} class MovieCollection < ForestAdminDatasourceToolkit::Collection def initialize(datasource) super(datasource, 'Movie') add_field('director', ManyToOneSchema.new( foreign_key: 'director_id', foreign_key_target: 'id', foreign_collection: 'People' )) end end ``` #### Query example The system can execute calls using both source and target collection fields: ```javascript Node.js theme={null} await dataSource.getCollection('movies').list( caller, { conditionTree: { aggregator: 'And', conditions: [ { field: 'title', operator: 'Equal', value: 'E.T.' }, { field: 'director:firstName', operator: 'Equal', value: 'Steven' }, ] }, sort: [{ field: 'director:birthDate', ascending: true }] }, ['id', 'title', 'director:firstName', 'director:lastName'] ); ``` ```ruby Ruby theme={null} datasource.collection('Movie').list( caller, ForestAdminDatasourceToolkit::Components::Query::Filter.new( condition_tree: ForestAdminDatasourceToolkit::Components::Query::ConditionTree::ConditionTreeBranch.new( aggregator: 'and', conditions: [ ForestAdminDatasourceToolkit::Components::Query::ConditionTree::ConditionTreeLeaf.new( field: 'title', operator: 'Equal', value: 'E.T.' ), ForestAdminDatasourceToolkit::Components::Query::ConditionTree::ConditionTreeLeaf.new( field: 'director:firstName', operator: 'Equal', value: 'Steven' ) ] ), sort: ForestAdminDatasourceToolkit::Components::Query::Sort.new( [{ field: 'director:birthDate', ascending: true }] ) ), ForestAdminDatasourceToolkit::Components::Query::Projection.new( ['id', 'title', 'director:firstName', 'director:lastName'] ) ) ``` #### Expected response structure ```json theme={null} { "id": 34, "title": "E.T", "director": { "firstName": "Steven", "lastName": "Spielberg" } } ``` ### Implementation scope Developers implementing your own datasources must handle prefixed field references in: * **Filters** (condition trees) * **Projections** (field selections) * **Aggregations** (calculation operations) Want to share your datasource with the community? Check out the [Forest experimental repository](https://github.com/ForestAdmin/forestadmin-experimental) to contribute. # Dummy datasource Source: https://docs.forest.app/get-started/connect/data-sources/dummy In-memory test datasource for development, testing, and demonstrations The Dummy datasource is an in-memory test datasource for **testing and learning Forest back-end functionality only**. It is not intended for production use. This datasource is strictly for development, testing, and learning purposes. All data is stored in memory and resets on application restart. ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createDummyDataSource } from '@forestadmin/datasource-dummy'; const agent = createAgent(options); agent.addDataSource(createDummyDataSource()); ``` ## Sample data The dummy datasource provides 4 pre-configured collections with realistic relationships: ### Collections **persons** (Authors) * `id` (Number, Primary Key) * `firstName` (String) * `lastName` (String) Sample: Edward O. Thorp, Isaac Asimov, Roberto Saviano, Stephen King **books** * `id` (Number, Primary Key) * `title` (String) * `publication` (Date) * `authorId` (Number) Sample: Beat the dealer, Foundation, Gomorrah, Misery, Christine, Running Man **libraries** * `id` (Number, Primary Key) * `name` (String) Sample: Mollat, Cultura, Amazon **librariesBooks** (Junction table) * `bookId` (Number, Primary Key) * `libraryId` (Number, Primary Key) ### Relationships The datasource demonstrates all common relationship types: * **One-to-Many**: persons → books (one author has many books) * **Many-to-One**: books → persons (many books have one author) * **Many-to-Many**: books ↔ libraries (via librariesBooks junction table) ## Features * **Full CRUD operations** - Create, read, update, and delete records * **Filtering** - 16+ filter operators (Contains, StartsWith, Equal, In, etc.) * **Sorting** - All columns are sortable * **Pagination** - Full pagination support * **Relationships** - Demonstrates 1-to-Many, Many-to-One, and Many-to-Many relationships ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin/datasource-dummy`](https://github.com/ForestAdmin/agent-nodejs/tree/main/packages/datasource-dummy). # Elasticsearch Source: https://docs.forest.app/get-started/connect/data-sources/elasticsearch Connect to Elasticsearch indices with query and aggregation support Only available for Node.js. The Elasticsearch datasource connects to Elasticsearch indices, allowing you to query, filter, and aggregate your data through Forest. ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createElasticsearchDataSource } from '@forestadmin-experimental/datasource-elasticsearch'; const agent = createAgent(options); agent.addDataSource( createElasticsearchDataSource({ node: 'http://localhost:9200', auth: { username: 'elastic', password: 'your-password' } }) ); ``` ## Configuration options ### Index-based collections Map each Elasticsearch index to a Forest collection: ```javascript theme={null} createElasticsearchDataSource({ node: 'http://localhost:9200', indices: [ { name: 'products', fields: { id: { type: 'Number' }, name: { type: 'String' }, price: { type: 'Number' }, category: { type: 'String' }, tags: { type: 'StringList' }, created_at: { type: 'Date' } } } ] }) ``` ### Template-based collections Use Elasticsearch index templates to automatically generate collections: ```javascript theme={null} createElasticsearchDataSource({ node: 'http://localhost:9200', templates: [ { name: 'logs-*', fields: { timestamp: { type: 'Date' }, level: { type: 'String' }, message: { type: 'String' } } } ] }) ``` ### Authentication The datasource supports multiple authentication methods: ```javascript theme={null} // Basic authentication createElasticsearchDataSource({ node: 'https://elastic.example.com:9200', auth: { username: 'elastic', password: 'password' } }) // API key authentication createElasticsearchDataSource({ node: 'https://elastic.example.com:9200', auth: { apiKey: 'your-api-key' } }) // Bearer token createElasticsearchDataSource({ node: 'https://elastic.example.com:9200', auth: { bearer: 'your-bearer-token' } }) ``` ### SSL/TLS configuration ```javascript theme={null} createElasticsearchDataSource({ node: 'https://elastic.example.com:9200', tls: { ca: fs.readFileSync('./ca.crt'), rejectUnauthorized: true } }) ``` ## Field types The datasource supports the following field types: * `String` - Text fields * `Number` - Numeric fields * `Boolean` - Boolean fields * `Date` - Date/datetime fields * `StringList` - Array of strings * `Json` - Nested objects (stored as JSON) Arrays must be explicitly specified using the `List` suffix (e.g., `StringList`). ## Query capabilities ### Filtering The datasource supports 16+ filter operators: * `Equal`, `NotEqual` * `GreaterThan`, `LessThan` * `In`, `NotIn` * `Contains`, `StartsWith`, `EndsWith` * `Present`, `Blank` * `Match` (full-text search) * And more ### Sorting Sort records by any field: ```javascript theme={null} // Forest UI automatically generates queries with sorting // Example: GET /products?sort=-price (descending by price) ``` ### Aggregations The datasource supports Elasticsearch aggregations for charts and analytics: * Count * Sum * Average * Min/Max * Terms aggregation (group by) ### Native SQL queries Execute Elasticsearch SQL queries directly: ```javascript theme={null} agent.customizeCollection('products', collection => { collection.addAction('Run SQL Query', { scope: 'Global', execute: async (context) => { const result = await context.dataSource.executeNativeQuery( 'SELECT category, AVG(price) FROM products GROUP BY category' ); return result; } }); }); ``` ## Live Query This datasource supports Live Query for advanced filtering and segmentation. [Learn more about Live Queries](/product/process/segments/smart-segments/overview) ## Limitations * **No joins** - Relationships between indices are not supported * **No object sub-models** - Nested objects are flattened to JSON fields * **No geospatial types** - Point fields are not supported * **Array specification** - Arrays must be manually specified with `List` suffix * **Read-heavy** - Optimized for search and analytics, not transactional workloads ## Elasticsearch 8.x support This datasource is built for Elasticsearch 8.x and uses the official `@elastic/elasticsearch` client. It supports: * Index templates * Data streams * Modern query DSL * SQL API * Security features (authentication, TLS) ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin-experimental/datasource-elasticsearch`](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-elasticsearch). # GraphQL / Hasura Source: https://docs.forest.app/get-started/connect/data-sources/graphql-hasura Connect GraphQL or Hasura to Forest Only available for Node.js. The GraphQL / Hasura datasource connects your GraphQL API or Hasura instance to Forest, allowing you to browse and manage your data through your back-office without writing custom integration code. ## Installation ```bash theme={null} npm install @forestadmin-experimental/datasource-graphql-hasura ``` ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createGraphqlHasuraDataSource } from '@forestadmin-experimental/datasource-graphql-hasura'; const agent = createAgent(options); agent.addDataSource( createGraphqlHasuraDataSource({ url: process.env.GRAPHQL_ENDPOINT, headers: { 'x-hasura-admin-secret': process.env.HASURA_ADMIN_SECRET } }) ); ``` ## Source code [github.com/ForestAdmin/forestadmin-experimental, datasource-graphql-hasura](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-graphql-hasura) ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin-experimental/datasource-graphql-hasura`](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-graphql-hasura). # Hubspot Source: https://docs.forest.app/get-started/connect/data-sources/hubspot Connect to Hubspot CRM data including contacts, companies, deals, and tickets Only available for Node.js. This is a read-only datasource. The Hubspot datasource connects to your Hubspot CRM through the HubSpot API, allowing you to browse and search contacts, companies, deals, tickets, and other CRM objects in Forest. ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createHubSpotDataSource } from '@forestadmin-experimental/datasource-hubspot-translation'; const agent = createAgent(options); agent.addDataSource( createHubSpotDataSource({ accessToken: process.env.HUBSPOT_ACCESS_TOKEN }) ); ``` ## Configuration options ### Access token Create a Private App in HubSpot to get an access token: 1. Go to Settings → Integrations → Private Apps in your HubSpot account 2. Create a new Private App 3. Grant necessary scopes (contacts, companies, deals, tickets, etc.) 4. Copy the access token ```javascript theme={null} createHubSpotDataSource({ accessToken: process.env.HUBSPOT_ACCESS_TOKEN }) ``` ### Rate limiting The datasource includes built-in rate limiting using Bottleneck: ```javascript theme={null} createHubSpotDataSource({ accessToken: process.env.HUBSPOT_ACCESS_TOKEN, rateLimiting: { maxConcurrent: 10, // Maximum concurrent requests minTime: 100 // Minimum time between requests (ms) } }) ``` Default rate limits respect HubSpot API constraints: * Search API: 4 requests per second * Standard API: 100 requests per 10 seconds ### Custom object configuration Include custom HubSpot objects in your Forest: ```javascript theme={null} createHubSpotDataSource({ accessToken: process.env.HUBSPOT_ACCESS_TOKEN, customObjects: ['2-12345678'] // Custom object IDs }) ``` ## Supported objects The datasource automatically includes these standard HubSpot objects: * **companies** - Company records * **contacts** - Contact records * **deals** - Deal pipeline records * **line\_items** - Product line items * **products** - Product catalog * **quotes** - Quote records * **tickets** - Support ticket records * **owners** - HubSpot user accounts Custom objects can be added via configuration. ## Read-only datasource This datasource is **read-only**. You can: * Browse records * Search and filter * View relationships * Export data You cannot: * Create new records * Update existing records * Delete records For write operations, contact us for more info. ## Filtering capabilities The datasource supports filtering with these operators: * `Equal` - Exact match * `NotEqual` - Exclude matches * `In` - Match any value in list * `LessThan`, `GreaterThan` - Numeric/date comparisons * `Contains` - Text contains (limited support) **Limitation:** Maximum 5 filter criteria per query due to HubSpot Search API constraints. ## Relationships The datasource attempts to map HubSpot associations to Forest relationships: ```javascript theme={null} // Example: Companies associated with Contacts // Automatically exposed as relationships in Forest ``` However, relationships are limited: * **No native relation support** - Relationships are flattened or require manual configuration * **Association API constraints** - Complex associations may not work correctly ## Pagination Browse large datasets with automatic pagination: * Default page size: 100 records * Maximum page size: 100 records (HubSpot API limit) * Automatic "Load More" in Forest UI ## Rate limiting and performance The datasource handles HubSpot rate limits automatically: **Search API limits:** * 4 requests per second * Used for filtering and searching **Standard API limits:** * 100 requests per 10 seconds * Used for basic CRUD operations If you exceed rate limits, the datasource will queue requests and retry automatically. ## Live Query This datasource supports Live Query for advanced filtering and segmentation. [Learn more about Live Queries](/product/process/segments/smart-segments/overview) ## Limitations * **Read-only** - No create, update, or delete operations * **No native relations** - Relationships require manual configuration * **Search constraints** - Maximum 5 filter criteria per query * **Rate limiting** - Subject to HubSpot API rate limits * **No real-time sync** - Data is fetched on-demand, not cached * **Association complexity** - Complex multi-level associations may not work ## HubSpot API token scopes Your Private App token needs these scopes: * `crm.objects.companies.read` * `crm.objects.contacts.read` * `crm.objects.deals.read` * `crm.objects.line_items.read` * `crm.objects.quotes.read` * `tickets` * `crm.objects.owners.read` For custom objects, add: * `crm.objects.custom.read` ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin-experimental/datasource-hubspot-translation`](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-hubspot-translation). # MongoDB datasource Source: https://docs.forest.app/get-started/connect/data-sources/mongodb Connect directly to MongoDB without an ORM using automatic schema introspection The MongoDB datasource connects directly to MongoDB without requiring Mongoose or another ORM. It automatically introspects your collections by sampling documents to infer the schema. MongoDB datasource is only available for Node.js. For Ruby, check the [Mongoid datasource](/get-started/connect/data-sources/mongoid). ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createMongoDataSource } from '@forestadmin/datasource-mongo'; const agent = createAgent(options); agent.addDataSource( createMongoDataSource({ uri: 'mongodb://localhost:27017', dataSource: { flattenMode: 'auto' } }) ); ``` ## Schema introspection Since MongoDB lacks a predefined schema, the back-end samples documents from each collection during startup to infer the structure. You can control this behavior: ```javascript theme={null} createMongoDataSource({ uri: 'mongodb://localhost:27017', introspection: { collectionSampleSize: 100, // Documents sampled per collection referenceSampleSize: 10, // References analyzed for relationships maxPropertiesPerObject: 30 // Maximum properties extracted as columns } }) ``` Setting sample sizes too high may slow back-end startup times. ## Filtering collections Exclude specific collections from Forest: ```javascript theme={null} const agent = createAgent(options); agent.addDataSource( createMongoDataSource({ uri: connectionString }), { exclude: ['accounts', 'accounts_bills', 'accounts_bills_items'] } ); ``` ## Flattening nested data MongoDB stores nested BSON documents. Configure flattening to convert nested structures into columns or separate collections: ```javascript theme={null} createMongoDataSource({ uri: 'mongodb://localhost:27017', dataSource: { flattenMode: 'manual', flattenOptions: { persons: { asModels: ['bills'], // Convert to separate collections asFields: ['address'] // Move to root level // or: asFields: ['address.city', 'address.country'] // or: asFields: [{ field: 'address', level: 1 }] } } } }) ``` ## Data navigation When customizing your back-end, navigate paths using these separators: * **Nested fields**: Use `@@@` to access nested properties * **Related data**: Use `:` to navigate relationships **Example**: `address:city@@@name` accesses the name field within city within the address relation. ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin/datasource-mongo`](https://github.com/ForestAdmin/agent-nodejs/tree/main/packages/datasource-mongo). # Mongoid datasource Source: https://docs.forest.app/get-started/connect/data-sources/mongoid Connect Forest to MongoDB using Mongoid models with flexible nested document handling The Mongoid datasource imports collections from a Mongoid instance into Forest, with configurable strategies for handling deeply nested embedded documents. Mongoid datasource is only available for Ruby. For Node.js, check the [Mongoose datasource](/get-started/connect/data-sources/mongoose) or [MongoDB datasource](/get-started/connect/data-sources/mongodb). ## Basic usage ```ruby theme={null} require 'forest_admin_datasource_mongoid' datasource = ForestAdminDatasourceMongoid::Datasource.new( options: { flatten_mode: 'auto', } ) ``` ## Flatten modes The Mongoid datasource provides three modes for handling embedded documents: ### `flatten_mode: 'auto'` Embedded documents (embeds\_one, embeds\_many) are automatically converted into separate Forest collections. ```ruby theme={null} datasource = ForestAdminDatasourceMongoid::Datasource.new( options: { flatten_mode: 'auto', } ) ``` ### `flatten_mode: 'manual'` You control which virtual collections are created and which fields are moved to the root level. ```ruby theme={null} datasource = ForestAdminDatasourceMongoid::Datasource.new( options: { flatten_mode: 'manual', # Configure which paths become collections vs fields # Contact Forest support for manual mode configuration details } ) ``` ### `flatten_mode: 'none'` Mongoid models are displayed as-is, with embedded objects appearing as raw JSON. ```ruby theme={null} datasource = ForestAdminDatasourceMongoid::Datasource.new( options: { flatten_mode: 'none', } ) ``` ## Data navigation When working with nested or related data, use specific separators: * **Nested fields**: Use `@@@` to access nested properties * **Related data**: Use `:` to navigate relationships **Example**: `address:city@@@name` accesses the name field within city within the address relation. ## Source code This connector is open source. Browse the code or contribute on GitHub: [`forest_admin_datasource_mongoid`](https://github.com/ForestAdmin/agent-ruby/tree/main/packages/forest_admin_datasource_mongoid). # Mongoose datasource Source: https://docs.forest.app/get-started/connect/data-sources/mongoose Connect to MongoDB using Mongoose ODM with flexible data transformation strategies The Mongoose datasource connects to MongoDB through Mongoose models, automatically importing your collections into Forest with support for nested objects, arrays, and relationships. Mongoose datasource is only available for Node.js. For Ruby, check the [Mongoid datasource](/get-started/connect/data-sources/mongoid). ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createMongooseDataSource } from '@forestadmin/datasource-mongoose'; import connection from './mongoose-models'; const agent = createAgent(options); agent.addDataSource( createMongooseDataSource(connection, { flattenMode: 'none' }) ); ``` ## Example schema Here's an example Mongoose schema with nested data: ```javascript theme={null} import mongoose from 'mongoose'; const personSchema = new mongoose.Schema({ name: String, age: Number, address: { streetName: String, city: String, country: String }, bills: [{ title: String, amount: Number, issueDate: Date, paidBy: [String] }] }); const Person = mongoose.model('persons', personSchema); ``` ## Flatten modes The Mongoose datasource offers four transformation strategies: ### `flattenMode: 'auto'` Arrays of objects and references are converted to independent collections. Other nested fields are moved to the root level. ```javascript theme={null} createMongooseDataSource(connection, { flattenMode: 'auto' }) ``` ### `flattenMode: 'none'` No transformations are made. Forest collections use the exact same structure as your Mongoose models. ```javascript theme={null} createMongooseDataSource(connection, { flattenMode: 'none' }) ``` ### `flattenMode: 'manual'` You are in full control of which virtual collections are created and which fields are moved to the root level. ```javascript theme={null} createMongooseDataSource(connection, { flattenMode: 'manual', asModels: { 'persons': ['bills'] // Convert bills array to separate collection }, asFields: { 'persons': ['address'] // Flatten address to root level } }) ``` ### `flattenMode: 'legacy'` Maintains backward compatibility with previous datasource versions. ```javascript theme={null} createMongooseDataSource(connection, { flattenMode: 'legacy' }) ``` ## Data navigation When working with nested or related data, use specific separators: * **Nested fields**: Use `@@@` to access nested properties * **Related data**: Use `:` to navigate relationships **Example**: `address:city@@@name` accesses the name field within city within the address relation. ## Virtual collections In `auto` mode, the datasource may create virtual collections to represent relationships. If you use a collection whitelist, make sure to include these virtual collections in your configuration. ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin/datasource-mongoose`](https://github.com/ForestAdmin/agent-nodejs/tree/main/packages/datasource-mongoose). # Add datasources to your project Source: https://docs.forest.app/get-started/connect/data-sources/overview Connect one or multiple data sources - SQL databases, MongoDB, APIs, or custom connectors Forest connects to your data through datasources. You can add one or multiple datasources to your project, each representing a database, an API, or any data source you need to manage. ## Adding a datasource Use your back-end's method to add a datasource: ```javascript Node.js theme={null} import { createAgent } from '@forestadmin/agent'; import { createSqlDataSource } from '@forestadmin/datasource-sql'; const agent = createAgent(options); agent.addDataSource( createSqlDataSource('postgresql://user:pass@localhost:5432/mydb') ); ``` ```ruby Ruby theme={null} # app/lib/forest_admin_rails/create_agent.rb module ForestAdminRails class CreateAgent def self.setup! datasource = ForestAdminDatasourceActiveRecord::Datasource.new( Rails.configuration.database_configuration[Rails.env] ) @create_agent = ForestAdminAgent::Builder::AgentFactory.instance.add_datasource(datasource) customize @create_agent.build end def self.customize; end end end ``` Your data is now surfaced to Forest, allowing you to browse, search, edit, and manage your records. ## Adding multiple datasources You can connect multiple datasources to create a unified back-office across different databases, APIs, or systems: ```javascript Node.js theme={null} const agent = createAgent(options); // Primary PostgreSQL database agent.addDataSource( createSqlDataSource('postgresql://user:pass@localhost:5432/main') ); // MongoDB for analytics agent.addDataSource( createMongooseDataSource(mongoConnection) ); // External API agent.addDataSource( createCustomDataSource(apiConfig) ); ``` ```ruby Ruby theme={null} # app/lib/forest_admin_rails/create_agent.rb, inside CreateAgent.setup! factory = ForestAdminAgent::Builder::AgentFactory.instance # Primary PostgreSQL database factory.add_datasource( ForestAdminDatasourceActiveRecord::Datasource.new( Rails.configuration.database_configuration[Rails.env] ) ) # MongoDB for analytics factory.add_datasource(ForestAdminDatasourceMongoid::Datasource.new) @create_agent = factory ``` All collections from all datasources will appear in your Forest interface, giving you a unified view across your entire data landscape. ### Cross-database relationships You can define relationships between collections from different datasources: ```javascript Node.js theme={null} agent.customizeCollection('orders', collection => { collection.addManyToOneRelation('user', 'analytics_users', { foreignKey: 'user_id' }); }); ``` ```ruby Ruby theme={null} agent.customize_collection('orders') do |collection| collection.add_many_to_one_relation('user', 'analytics_users', foreign_key: 'user_id' ) end ``` This allows you to navigate between related data even when it lives in different databases or systems. [Learn more about relationships →](/product/process/relationships/overview) ## Available datasources * **[SQL](/get-started/connect/data-sources/sql)** - PostgreSQL, MySQL, MariaDB, SQL Server with automatic schema introspection * **[Sequelize](/get-started/connect/data-sources/sequelize)** - Connect through your existing Sequelize ORM models * **[Mongoose](/get-started/connect/data-sources/mongoose)** - MongoDB via Mongoose ODM with schema support * **[Dummy](/get-started/connect/data-sources/dummy)** - Generate fake data for testing and prototyping * **[Hubspot](/get-started/connect/data-sources/hubspot)** - Connect to Hubspot CRM (Contacts, Companies, Deals, Tickets) * **[Elasticsearch](/get-started/connect/data-sources/elasticsearch)** - Query Elasticsearch indices * **[Airtable](/get-started/connect/data-sources/airtable)** - Connect Airtable bases * **[Stripe](/get-started/connect/data-sources/stripe)** - Connect Stripe account data * **[RPC](/get-started/connect/data-sources/rpc)** - Connect remote data sources via RPC * **[GraphQL / Hasura](/get-started/connect/data-sources/graphql-hasura)** - Connect GraphQL APIs or Hasura instances * **[CosmosDB](/get-started/connect/data-sources/cosmosdb)** - Connect Azure CosmosDB ### Custom datasources Build custom datasources for proprietary systems, legacy databases, REST APIs, or any data source without an existing connector: * **[Replication](/get-started/connect/data-sources/custom-datasources/replication)** - Replicate external data into Forest * **[Translation](/get-started/connect/data-sources/custom-datasources/translation)** - Transform and adapt data from any source **Don't see the datasource you need?** [Contact us](https://www.forestadmin.com/contact) to request a datasource, and we'll help you connect it. # RPC Source: https://docs.forest.app/get-started/connect/data-sources/rpc Connect back-ends over RPC: expose a remote data source, or distribute one back-end across microservices The RPC datasource connects Forest back-ends to each other over RPC (Remote Procedure Call). Use it to expose a remote data source in your back-office, or to split one back-end across multiple services (microservices) that a main back-end aggregates into a single admin panel. ## Node.js In Node.js, the RPC datasource connects to a remote data source and exposes its data and operations in your back-office. ```bash theme={null} npm install @forestadmin-experimental/datasource-rpc ``` ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createRpcDataSource } from '@forestadmin-experimental/datasource-rpc'; const agent = createAgent(options); agent.addDataSource( createRpcDataSource({ url: process.env.RPC_SERVER_URL, }), ); ``` ## Ruby (distributed back-ends) In Ruby, the RPC datasource lets you distribute your Forest back-end across multiple microservices. Each service runs its own RPC back-end, and a **main back-end** aggregates them into a unified admin panel. **Why use RPC?** * **Microservices architecture**: your application is split into multiple services, each owning its data. * **Team isolation**: each team manages their own Forest configuration. * **Independent deployment**: update one service without redeploying the entire admin panel. * **Scalability**: distribute load across multiple agents. RPC architecture: a main agent connected to several RPC agents ### Main back-end The main back-end aggregates multiple RPC back-ends into a single Forest interface. It behaves exactly like a classic back-end with multiple datasources: once connected, you can define relations across datasources transparently. ```ruby theme={null} # Gemfile gem 'forest_admin_datasource_rpc' ``` ```ruby theme={null} # app/lib/forest_admin_rails/create_agent.rb module ForestAdminRails class CreateAgent def self.setup! @agent = ForestAdminAgent::Builder::AgentFactory.instance # Add RPC datasources @agent.add_datasource( ForestAdminDatasourceRpc.build(uri: 'http://customers-app:3002') ) @agent.add_datasource( ForestAdminDatasourceRpc.build( uri: 'http://billing-app:3003', auth_secret: 'YOUR-SHARED-AUTH-SECRET' ) ) # You can also add local datasources @agent.add_datasource(ForestAdminDatasourceMongoid.build) @agent.use(ForestAdminDatasourceRpc::ReconciliateRpc) @agent.build end end end ``` The `auth_secret` option is optional. If not specified, it defaults to your Forest project's `auth_secret`. Override it to use a different shared secret for RPC communication. The `ReconciliateRpc` plugin must be added **in the main back-end**. It relies on collection names matching across all back-ends, so if you rename collections in your RPC back-ends, keep all back-ends synchronized. #### Introspection caching By default, the main back-end introspects each RPC back-end at startup. Caching the introspection schema makes startup faster, lets the main back-end boot even if an RPC back-end is temporarily unavailable, and supports asynchronous deployments. RPC back-ends automatically generate a `.forestadmin-rpc-schema.json` file in development mode; pass it when adding a datasource: ```ruby theme={null} schema = JSON.parse( File.read(Rails.root.join('.forestadmin-rpc-schema.json')), symbolize_names: true ) @agent.add_datasource( ForestAdminDatasourceRpc.build( uri: 'http://customers-app:3002', introspection: schema ) ) ``` Regenerate the cached schema when an RPC back-end's collections change. ### RPC back-end An RPC back-end exposes its collections to the main back-end via the RPC protocol. ```ruby theme={null} # Gemfile gem 'forest_admin_rpc_agent' ``` Run the installation command with the `auth_secret` from your main Forest project: ```bash theme={null} forest_admin_rpc_agent install YOUR_AUTH_SECRET ``` This creates `config/initializers/forest_admin_rpc_agent.rb`, `lib/forest_admin_rpc_agent/create_rpc_agent.rb`, and mounts the RPC routes in `config/routes.rb`. The generated back-end looks like: ```ruby theme={null} module ForestAdminRpcAgent class CreateRpcAgent def self.setup! datasource = ForestAdminDatasourceActiveRecord::Datasource.new(Rails.env.to_sym) @agent = ForestAdminRpcAgent::Agent.instance.add_datasource(datasource) @agent.build end end end ``` You can use all standard customization methods (computed fields, actions, segments, etc.) on your RPC back-end collections. ### Cross-RPC relations When an RPC back-end needs to reference collections from another RPC back-end, import them by adding an RPC datasource that references the other back-end, then use `mark_collections_as_rpc: true` to indicate those collections are provisioned elsewhere. ```ruby theme={null} module ForestAdminRpcAgent class CreateRpcAgent def self.setup! # 1. Local datasource datasource = ForestAdminDatasourceActiveRecord::Datasource.new(Rails.env.to_sym) @agent = ForestAdminRpcAgent::Agent.instance.add_datasource(datasource) # 2. Remote RPC datasource, marked as RPC @agent.add_datasource( ForestAdminDatasourceRpc.build(uri: 'http://customers-app:3002'), mark_collections_as_rpc: true ) # 3. Cross-RPC relations (here in the Billing agent: # 'Invoice' is local, 'User' is imported from the Customers agent) @agent.customize_collection('Invoice') do |collection| collection.add_many_to_one_relation( 'user', 'User', { foreign_key: 'user_id', foreign_key_target: 'id' } ) end @agent.customize_collection('User') do |collection| collection.add_one_to_many_relation( 'invoices', 'Invoice', { origin_key: 'user_id', origin_key_target: 'id' } ) end @agent.build end end end ``` For finer control over which collections are marked, call `mark_collections_as_rpc` manually: ```ruby theme={null} # Mark specific collections @agent.mark_collections_as_rpc('User', 'Address') # Or use a regex to match a pattern of collections @agent.mark_collections_as_rpc(/^admin_/) ``` Supported cross-RPC relation types: `add_many_to_one_relation`, `add_one_to_many_relation`, `add_one_to_one_relation`, and `add_many_to_many_relation`. ## Source code This connector is open source. Browse the code or contribute on GitHub: * Node.js: [`@forestadmin-experimental/datasource-rpc`](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-rpc) * Ruby: [`forest_admin_datasource_rpc`](https://github.com/ForestAdmin/agent-ruby/tree/main/packages/forest_admin_datasource_rpc) # Sequelize datasource Source: https://docs.forest.app/get-started/connect/data-sources/sequelize Connect Forest to your existing Sequelize models with support for scopes, hooks, and associations The Sequelize datasource allows importing collections from a Sequelize instance, preserving your ORM configuration including scopes, hooks, associations, and validations. Sequelize datasource is only available for Node.js. For Ruby, check the [ActiveRecord datasource](/get-started/connect/data-sources/active-record). ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createSequelizeDataSource } from '@forestadmin/datasource-sequelize'; import { Sequelize, Model, DataTypes } from 'sequelize'; // Initialize Sequelize const sequelize = new Sequelize('postgresql://user:pass@localhost:5432/mydb'); // Define your models class User extends Model {} User.init( { username: DataTypes.STRING, birthday: DataTypes.DATE, }, { sequelize, modelName: 'user' } ); // Add to agent const agent = createAgent(options); agent.addDataSource(createSequelizeDataSource(sequelize)); ``` ## Features The Sequelize datasource automatically preserves your ORM configuration: * **Sequelize scopes** - Mapped to Forest segments * **Sequelize hooks** - Continue to execute as configured * **Associations** - Relationships are automatically recognized * **Validations** - Model validations are enforced ## Live Query support Enable SQL-based reporting by setting a connection identifier: ```javascript theme={null} agent.addDataSource( createSequelizeDataSource(sequelize, { liveQueryConnections: 'main_database' }) ); ``` This allows authorized users to create Live Query charts, analytics, and segments that execute custom SQL directly against your database. Live Queries execute raw SQL. Ensure proper access controls and review queries before deploying to production. ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin/datasource-sequelize`](https://github.com/ForestAdmin/agent-nodejs/tree/main/packages/datasource-sequelize). # Snowflake Source: https://docs.forest.app/get-started/connect/data-sources/snowflake Connect Snowflake tables and views to Forest (read-only, via ODBC) The Snowflake data source allows importing tables and views from a Snowflake account into Forest via ODBC. It does **not** rely on ActiveRecord, Forest connects to Snowflake directly through the `ruby-odbc` driver, translates Forest filters/projections/aggregations into parameterised SQL, and streams the results back as plain Ruby objects. This data source is **read-only**. Every Forest column is emitted with `is_read_only: true`, so the schema emitter sets the collection-level `isReadOnly: true` and the UI hides create/edit/delete actions. Direct calls to `create`, `update`, or `delete` raise a `ForestException` with an explicit read-only message as a defence-in-depth guard. ## Installation To make everything work as expected, you need to: * install the gem `forest_admin_datasource_snowflake`. * have the **unixODBC** system library and the **Snowflake ODBC driver** installed on the host running the agent. On Ubuntu/Debian: `apt-get install unixodbc-dev`, then install [Snowflake's ODBC driver](https://docs.snowflake.com/en/developer-guide/odbc/odbc) and reference it from your `odbcinst.ini`. ## Usage Register the datasource from inside the back-end's setup hook (`ForestAdminRails::CreateAgent.setup!` for Rails apps), the same place where you would register an Active Record or Mongoid source: ```ruby theme={null} module ForestAdminRails class CreateAgent def self.setup! datasource = ForestAdminDatasourceSnowflake::Datasource.new( conn_str: "DRIVER={Snowflake};" \ "Server=#{ENV.fetch('SNOWFLAKE_ACCOUNT')}.snowflakecomputing.com;" \ "UID=#{ENV.fetch('SNOWFLAKE_USER')};" \ "PWD=#{ENV.fetch('SNOWFLAKE_PASSWORD')};" \ "Warehouse=#{ENV.fetch('SNOWFLAKE_WAREHOUSE')};" \ "Database=#{ENV.fetch('SNOWFLAKE_DATABASE')};" \ "Schema=#{ENV.fetch('SNOWFLAKE_SCHEMA', 'PUBLIC')}" ) @create_agent = ForestAdminAgent::Builder::AgentFactory.instance.add_datasource(datasource, {}) customize @create_agent.build end end end ``` The snippets below that pass options to `add_datasource` (e.g. `include:` / `exclude:`) drop into the same `setup!` body in place of the `add_datasource(datasource, {})` call. ### Authentication The data source doesn't interpret authentication options, `conn_str` is parsed as `key=value;...` and the resulting attributes are handed straight to the Snowflake ODBC driver. Any [parameter the driver accepts](https://docs.snowflake.com/en/developer-guide/odbc/odbc-parameters) works, including the production-friendly key-pair / JWT flow: ```ruby theme={null} conn_str: "DRIVER={Snowflake};" \ "Server=#{ENV.fetch('SNOWFLAKE_ACCOUNT')}.snowflakecomputing.com;" \ "UID=#{ENV.fetch('SNOWFLAKE_USER')};" \ "AUTHENTICATOR=SNOWFLAKE_JWT;" \ "PRIV_KEY_FILE=#{ENV.fetch('SNOWFLAKE_PRIV_KEY_FILE')};" \ # Optional, only when the private key is encrypted: "PRIV_KEY_FILE_PWD=#{ENV.fetch('SNOWFLAKE_PRIV_KEY_PWD')};" \ "Warehouse=#{ENV.fetch('SNOWFLAKE_WAREHOUSE')};" \ "Database=#{ENV.fetch('SNOWFLAKE_DATABASE')};" \ "Schema=#{ENV.fetch('SNOWFLAKE_SCHEMA', 'PUBLIC')}" ``` The back-end process must have read access to the `PRIV_KEY_FILE` path. `EXTERNALBROWSER` (SSO) and `OAUTH` flows work the same way, set the relevant `AUTHENTICATOR=` and supporting parameters per the driver docs. ## Automatic schema discovery By default, every user-schema, non-system table reachable by the configured Snowflake user is exposed as a Forest collection. System tables and the `INFORMATION_SCHEMA` views are filtered out automatically. At boot, the data source issues a small fixed set of metadata queries, independent of how many tables you expose: * one ODBC `tables` call to enumerate the readable tables and views. * one bulk `INFORMATION_SCHEMA.COLUMNS` query that returns every column for the schema in a single round-trip. Each Forest collection reads its slice from the pre-fetched result, so introspection cost no longer scales with table count. * one `SHOW PRIMARY KEYS IN SCHEMA` query to recover declared primary keys (composite keys preserved, ordered by `key_sequence`). * one `SHOW IMPORTED KEYS IN SCHEMA` query to recover declared foreign keys (see [Foreign-key auto-discovery](#foreign-key-auto-discovery)). The primary key for each collection is resolved in the following order: 1. an operator-supplied [`primary_keys:`](#overriding-the-primary-key) override, 2. any Snowflake-declared primary key (via `SHOW PRIMARY KEYS IN SCHEMA`), 3. a column literally named `id` (case-insensitive), 4. the first column as a last resort. Snowflake doesn't expose primary key information through ODBC's standard column metadata, hence the multi-step resolution. If any of the metadata queries fail (typically because the connecting role lacks the privilege), the failure is logged to stderr with a `[forest_admin_datasource_snowflake]` prefix and skipped. The result is cached so the broken query isn't re-issued on every collection lookup. ### Restricting the imported tables Use the standard agent-level `include:` / `exclude:` options when registering the datasource. The data source itself exposes every readable user-schema table; the back-end decides which ones to publish. ```ruby theme={null} ForestAdminAgent::Builder::AgentFactory.instance.add_datasource( ForestAdminDatasourceSnowflake::Datasource.new(conn_str: ENV.fetch('SNOWFLAKE_CONN_STR')), include: ['BILLING_USAGE', 'USAGE_ANOMALIES', 'CURRENCY_RATES'] ) # or, equivalently: ForestAdminAgent::Builder::AgentFactory.instance.add_datasource( ForestAdminDatasourceSnowflake::Datasource.new(conn_str: ENV.fetch('SNOWFLAKE_CONN_STR')), exclude: ['INTERNAL_LOG'] ) ``` This is the same pattern used by the other Forest data sources, so collection filtering stays consistent across your agent. ### Targeting a specific schema A datasource instance always represents a **single Snowflake schema**, Forest collection names are unqualified, so two tables with the same name in different schemas would collide. To expose tables from multiple schemas, instantiate one datasource per schema. The active schema is resolved as follows: * if `Schema=` is set in the connection string, it wins. The datasource parses it (case-insensitive) at construction time and issues `USE SCHEMA ""` on every new connection so the session, the table-list filter, and all introspection queries stay aligned. * if `Schema=` is omitted, the datasource snapshots `CURRENT_SCHEMA()` once at boot (whatever default the Snowflake user/role exposes) and uses that as the active schema for the rest of its lifetime. * if `Schema=` is omitted **and** `CURRENT_SCHEMA()` is null (the role has no default), the datasource raises `ForestAdminDatasourceSnowflake::Error` at boot with a message asking you to set `Schema=` explicitly. ```ruby theme={null} ForestAdminDatasourceSnowflake::Datasource.new( conn_str: "DRIVER={Snowflake};...;Schema=ANALYTICS" ) ``` ### Overriding the primary key For tables where the primary key cannot be auto-resolved, for example, a table without a Snowflake-declared PK, no `id` column, and where the first column isn't really the key, pass an explicit `primary_keys:` mapping. The lookup is case-insensitive on the table name. Pass an array to declare a composite key. ```ruby theme={null} ForestAdminDatasourceSnowflake::Datasource.new( conn_str: ENV.fetch('SNOWFLAKE_CONN_STR'), primary_keys: { 'CUSTOMER_EVENTS' => 'EVENT_UUID', 'orders' => 'ORDER_ID', 'usage_quotas' => %w[CUSTOMER_ID EVENT_TYPE] } ) ``` This override sits at the top of the resolution chain and takes precedence over any Snowflake-declared PK or fallback. If a declared column name does not match any column on the target table, the data source raises `ForestAdminDatasourceSnowflake::Error` at boot, silent fallback would otherwise mask configuration typos. ## Type mapping Column types are resolved from the Snowflake-native `DATA_TYPE` returned by `INFORMATION_SCHEMA.COLUMNS`. | Snowflake type | Forest type | | -------------------------------------------------------------------------------------------- | ----------- | | `BOOLEAN` | `Boolean` | | `NUMBER`, `DECIMAL`, `NUMERIC`, `INT`, `INTEGER`, `BIGINT`, `SMALLINT`, `TINYINT`, `BYTEINT` | `Number` | | `FLOAT`, `FLOAT4`, `FLOAT8`, `DOUBLE`, `DOUBLE PRECISION`, `REAL` | `Number` | | `VARCHAR`, `CHAR`, `CHARACTER`, `STRING`, `TEXT` | `String` | | `DATE` | `Dateonly` | | `TIME` | `Time` | | `DATETIME`, `TIMESTAMP`, `TIMESTAMP_NTZ`, `TIMESTAMP_LTZ`, `TIMESTAMP_TZ` | `Date` | | `VARIANT`, `OBJECT`, `ARRAY` | `Json` | | `BINARY`, `VARBINARY` | `Binary` | | `GEOGRAPHY`, `GEOMETRY`, `VECTOR` | `String` | Any type not in the table above falls back to `String`. `VARIANT` / `OBJECT` / `ARRAY` columns are JSON-parsed at projection time so the Forest UI receives structured data, not raw strings. `TIMESTAMP_LTZ` and `TIMESTAMP_TZ` are normalised at the session level via `ALTER SESSION SET TIMEZONE = 'UTC'`, so all three TIMESTAMP variants serialise consistently as UTC. This avoids subtle bugs where rows render with different offsets depending on the column variant. ## Foreign-key auto-discovery Snowflake foreign keys are not enforced by the engine, they exist purely as documentation. If you have defined them in your warehouse, the data source picks them up automatically at boot and exposes them as Forest `ManyToOne` relations. Discovery runs unconditionally: a `SHOW IMPORTED KEYS IN SCHEMA` query at boot adds a relation field on the source collection for each FK it returns. The relation name is `{source_column}_{target_table}` (downcased). If the introspection query fails (typically because the connecting role lacks the privilege), the failure is logged and skipped, the rest of the data source remains usable. Auto-discovery only handles relations defined **inside Snowflake**. Cross-data-source relations, for example a Snowflake `BILLING_USAGE.CUSTOMER_ID` pointing at a Postgres `customers.id`, cannot be discovered (Snowflake has no concept of an FK to another database). Wire those manually in the back-end layer with `add_many_to_one_relation` / `add_one_to_many_relation`. ## Connection pool The data source uses [`connection_pool`](https://github.com/mperham/connection_pool) under the hood. By default the pool is sized at 5 with a 5-second checkout timeout. Tune via `pool_size:` and `pool_timeout:` if you expect concurrent traffic. ```ruby theme={null} ForestAdminDatasourceSnowflake::Datasource.new( conn_str: ENV.fetch('SNOWFLAKE_CONN_STR'), pool_size: 10, pool_timeout: 10 ) ``` `with_connection` automatically retries the block once after a connection-lost ODBC error (communication failures, expired sessions, expired Snowflake auth tokens, etc.), cycling the pool between attempts so stale handles get closed before re-checkout. Persistent failures bubble up to the caller. ## Statement timeout To cap any single Forest-driven Snowflake query, pass `statement_timeout:` (in seconds). The data source issues `ALTER SESSION SET STATEMENT_TIMEOUT_IN_SECONDS = N` on each new connection. ```ruby theme={null} ForestAdminDatasourceSnowflake::Datasource.new( conn_str: ENV.fetch('SNOWFLAKE_CONN_STR'), statement_timeout: 60 ) ``` This is recommended in production: a runaway aggregate query won't be able to pin a pool slot indefinitely. ## Full reference ```ruby theme={null} ForestAdminDatasourceSnowflake::Datasource.new( # Required: ODBC connection string, DRIVER, Server, UID, PWD, Warehouse, # Database, Schema, plus any Snowflake-specific options. conn_str: ENV.fetch('SNOWFLAKE_CONN_STR'), # Optional: explicit primary key per table when auto-resolution can't find the # right one. Case-insensitive on the table name. Pass an array for composite # keys. Sits at the top of the resolution chain (above SHOW PRIMARY KEYS, # the 'id' column, and the first-column fallback). Raises at boot if a # declared column name doesn't match any column on the table. primary_keys: { 'orders' => 'ORDER_ID', 'usage_quotas' => %w[CUSTOMER_ID EVENT_TYPE] }, # Optional: connection pool tuning. Defaults: 5 connections, 5s checkout timeout. pool_size: 5, pool_timeout: 5, # Optional: cap any one query at N seconds via ALTER SESSION. statement_timeout: 60 ) ``` ## Source code This connector is open source. Browse the code or contribute on GitHub: [`forest_admin_datasource_snowflake`](https://github.com/ForestAdmin/agent-ruby/tree/main/packages/forest_admin_datasource_snowflake). # SQL datasource Source: https://docs.forest.app/get-started/connect/data-sources/sql Connect to PostgreSQL, MySQL, MariaDB, or SQL Server with automatic schema introspection This datasource is only available for Node.js agents. The SQL datasource connects directly to SQL databases with automatic schema introspection. Each database table or view maps to a Forest collection - tables offer full read-write capabilities while views are read-only. ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createSqlDataSource } from '@forestadmin/datasource-sql'; const agent = createAgent(options); agent.addDataSource( createSqlDataSource({ uri: 'postgresql://user:pass@localhost:5432/mydb', sslMode: 'preferred' }) ); ``` ## Automatic schema introspection By default, the SQL datasource automatically discovers your database structure when the back-end starts. It extracts: * **Tables and views** - Each becomes a collection * **Columns** - With automatic type detection * **Primary keys** - For record identification * **Foreign keys** - Converted to relationships * **Indexes** - Used for query optimization This requires database credentials with access to `information_schema`. Ownership roles are recommended. Introspection is supported for PostgreSQL, MySQL, MariaDB, and Microsoft SQL Server. ## Configuration options ### Connection URI The connection URI format varies by database: ```javascript theme={null} createSqlDataSource({ uri: 'postgresql://user:password@host:5432/database' }) ``` ```javascript theme={null} createSqlDataSource({ uri: 'mysql://user:password@host:3306/database' }) ``` ```javascript theme={null} createSqlDataSource({ uri: 'mssql://user:password@host:1433/database' }) ``` ### SSL configuration Control SSL/TLS connection behavior: ```javascript theme={null} createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db', sslMode: 'preferred' // or 'verify', 'required', 'disabled', 'manual' }) ``` **SSL modes:** * `preferred` - Use SSL if available, otherwise unencrypted (default) * `required` - Require SSL, fail if unavailable * `verify` - Require SSL with certificate verification * `disabled` - Never use SSL * `manual` - Custom SSL configuration (advanced) ### Schema selection Specify which database schema to use (PostgreSQL, SQL Server): ```javascript theme={null} createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db', schema: 'my_schema' // default: 'public' }) ``` ### Connection pooling Configure connection pool for optimal performance: ```javascript theme={null} createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db', pool: { max: 20, // Maximum connections min: 5, // Minimum connections acquire: 30000, // Max time (ms) to get connection idle: 10000 // Close idle connections after 10s } }) ``` ### Read replicas Distribute read operations across replica databases: ```javascript theme={null} createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db', replication: { write: { host: 'primary.example.com' }, read: [ { host: 'replica1.example.com' }, { host: 'replica2.example.com' } ] } }) ``` ### SSH tunnel Connect through an SSH tunnel: ```javascript theme={null} createSqlDataSource({ uri: 'postgresql://user:pass@localhost:5432/db', ssh: { host: 'ssh-host.example.com', port: 22, username: 'ssh-user', privateKey: require('fs').readFileSync('/path/to/private-key') } }) ``` ### SOCKS5 proxy Route connections through a SOCKS5 proxy: ```javascript theme={null} createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db', proxySocks: { host: 'proxy.example.com', port: 1080, username: 'proxy-user', password: 'proxy-pass' } }) ``` ### Connection timeout Set maximum time to establish connection: ```javascript theme={null} createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db', connectionTimeoutInMs: 5000 // 5 seconds }) ``` ## Soft-deleted records Display records marked as deleted (soft deletes): ```javascript theme={null} // Show soft-deleted for specific collections createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db', displaySoftDeleted: ['users', 'projects'] }) // Show soft-deleted for all collections createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db', displaySoftDeleted: true }) ``` This is useful when your application uses soft deletes (e.g., `deleted_at` column) and you want to manage deleted records in Forest. ## Caching introspection The schema introspection is JSON serializable. Cache it to a file: ```javascript theme={null} import { createSqlDataSource } from '@forestadmin/datasource-sql'; import { writeFileSync, readFileSync } from 'fs'; // Option 1: Generate and cache schema const dataSource = await createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db' }); writeFileSync('./schema-cache.json', JSON.stringify(dataSource.schema)); // Option 2: Load from cache (faster startup, works offline) const schema = JSON.parse(readFileSync('./schema-cache.json', 'utf8')); const dataSource = createSqlDataSource({ uri: 'postgresql://user:pass@host:5432/db', schema: schema }); ``` **Benefits:** * Faster back-end startup (no introspection delay) * Work offline or with restricted credentials * Separate introspection and runtime credentials * Version control your schema ## Live Query support Enable SQL-based reporting by setting a connection identifier: ```javascript theme={null} createSqlDataSource({ uri: process.env.DATABASE_URL, liveQueryConnections: 'main_database' }) ``` This allows authorized users to create Live Query charts and segments that execute custom SQL directly against your database. Live Queries execute raw SQL. Ensure proper access controls and review queries before deploying to production. ## Supported databases | Database | Versions | Driver Package | Status | | -------------- | -------- | ------------------ | ---------------------------- | | **PostgreSQL** | 10+ | `pg` + `pg-hstore` | ✅ Full support (recommended) | | **MySQL** | 5.7+ | `mysql2` | ✅ Production-ready | | **MariaDB** | 10+ | `mariadb` | ✅ Production-ready | | **SQL Server** | 2017+ | `tedious` | ✅ Enterprise support | ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin/datasource-sql`](https://github.com/ForestAdmin/agent-nodejs/tree/main/packages/datasource-sql). # Stripe Source: https://docs.forest.app/get-started/connect/data-sources/stripe Connect Stripe to Forest Only available for Node.js. The Stripe datasource connects your Stripe account to Forest, allowing you to browse and manage your Stripe data (customers, payments, subscriptions, and more) directly from your back-office. ## Installation ```bash theme={null} npm install @forestadmin-experimental/datasource-stripe ``` ## Basic usage ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createStripeDataSource } from '@forestadmin-experimental/datasource-stripe'; const agent = createAgent(options); agent.addDataSource( createStripeDataSource({ secretKey: process.env.STRIPE_SECRET_KEY }) ); ``` ## Source code [github.com/ForestAdmin/forestadmin-experimental, datasource-stripe](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-stripe) ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin-experimental/datasource-stripe`](https://github.com/ForestAdmin/forestadmin-experimental/tree/main/packages/datasource-stripe). # Zendesk Source: https://docs.forest.app/get-started/connect/data-sources/zendesk Surface a Zendesk Support account (tickets, users, organizations) as Forest collections The Zendesk data source surfaces a Zendesk Support account as Forest collections. It exposes tickets, users and organizations (with each ticket carrying its comment thread inline) on top of the [Zendesk REST API](https://developer.zendesk.com/api-reference/ticketing/introduction/), so you can browse and edit them from your Forest project alongside your other data sources. This is the Zendesk **data source** (Zendesk data inside Forest). If you want to embed Forest data and actions inside Zendesk tickets instead, see the [Zendesk app](/product/embed/zendesk). ## Installation Install the package `@forestadmin/datasource-zendesk`. ```bash theme={null} yarn add @forestadmin/datasource-zendesk ``` ```javascript theme={null} import { createAgent } from '@forestadmin/agent'; import { createZendeskClient, createZendeskDataSource, } from '@forestadmin/datasource-zendesk'; const zendeskClient = createZendeskClient({ subdomain: process.env.ZENDESK_SUBDOMAIN, email: process.env.ZENDESK_EMAIL, apiToken: process.env.ZENDESK_API_TOKEN, }); const agent = createAgent(options).addDataSource( createZendeskDataSource({ client: zendeskClient }), ); ``` Sharing the same `zendeskClient` instance with the [Zendesk plugins](/product/process/advanced-concepts/plugins/zendesk) keeps auth, base URL and best-effort logger consistent across calls. It is recommended, but not required — the plugins can also be configured with raw credentials directly. Install the gem `forest_admin_datasource_zendesk`. ```ruby theme={null} module ForestAdminRails class CreateAgent def self.setup! datasource = ForestAdminDatasourceZendesk::Datasource.new( subdomain: ENV['ZENDESK_SUBDOMAIN'], username: ENV['ZENDESK_USERNAME'], token: ENV['ZENDESK_API_TOKEN'] ) @create_agent = ForestAdminAgent::Builder::AgentFactory.instance.add_datasource(datasource, {}) customize @create_agent.build end end end ``` ## Configuration The datasource authenticates against Zendesk using an [API token](https://support.zendesk.com/hc/en-us/articles/4408889192858-Managing-access-to-the-Zendesk-API). `createZendeskDataSource` accepts a `ZendeskClientProvider` — either a pre-built `client`, or the three raw credentials below (which the factory then uses to build one). The two shapes are mutually exclusive. When the credentials are passed directly, the client constructor throws `ZendeskConfigurationError` (a `ValidationError`) if any of them is missing or blank. | Option | Description | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | `client` | A `ZendeskClient` instance built with `createZendeskClient`. **Required when `subdomain` / `email` / `apiToken` are not provided.** | | `subdomain` | The subdomain of your Zendesk account (e.g. `acme` for `https://acme.zendesk.com`). **Required when `client` is not provided.** | | `email` | The email address associated with the API token (typically a Zendesk admin/agent account). Required alongside `subdomain` and `apiToken`. | | `apiToken` | A Zendesk API token generated from `Admin Center → Apps and integrations → APIs → Zendesk API`. Required alongside `subdomain` and `email`. | Sharing one pre-built client with the plugins is the recommended pattern (single auth setup, single logger); letting the factory build it for you is convenient when the plugins use raw credentials too: ```javascript theme={null} // Recommended — share the same client with the plugins createZendeskDataSource({ client: createZendeskClient({ subdomain, email, apiToken }), }); // Or let the factory build the client for you createZendeskDataSource({ subdomain, email, apiToken }); ``` All three options are mandatory; the back-end fails fast with a `ForestAdminDatasourceZendesk::ConfigurationError` if any of them is missing or blank. | Option | Description | | ----------- | ----------------------------------------------------------------------------------------------- | | `subdomain` | The subdomain of your Zendesk account (e.g. `acme` for `https://acme.zendesk.com`). | | `username` | The email address associated with the API token (typically a Zendesk admin/agent account). | | `token` | A Zendesk API token generated from `Admin Center → Apps and integrations → APIs → Zendesk API`. | The `forest_admin_datasource_zendesk` package also ships two action plugins (`CreateTicketWithNotification` and `CloseTicket`) that you can attach to any host collection. See [the Zendesk plugins page](/product/process/advanced-concepts/plugins/zendesk) for details. ## Provided collections Once the data source is registered, three collections are added to your Forest project: | Collection | Primary endpoint | Notes | | ---------------------- | ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | `zendesk_ticket` | Search API (`/api/v2/search.json?query=type:ticket`) | Full read/write. Embeds `requester`, `assignee`, `organization` and an inline `comments` thread. | | `zendesk_user` | Search API (`/api/v2/search.json?query=type:user`) | Full read/write. | | `zendesk_organization` | Search API (`/api/v2/search.json?query=type:organization`) | Full read/write. | The exact collection names are exported as the `COLLECTION_NAMES` constant — use it instead of hard-coding the strings when you customize the collections: ```javascript theme={null} import { COLLECTION_NAMES } from '@forestadmin/datasource-zendesk'; agent.customizeCollection(COLLECTION_NAMES.ticket, collection => { /* ... */ }); ``` | Collection | Primary endpoint | Notes | | --------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------ | | `ZendeskTicket` | Search API (`/api/v2/search?query=type:ticket`) | Full read/write. Embeds `requester`, `assignee`, `organization` and an inline `comments` thread. | | `ZendeskUser` | Search API (`/api/v2/search?query=type:user`) | Full read/write. | | `ZendeskOrganization` | Search API (`/api/v2/search?query=type:organization`) | Full read/write. | ### Relationships The following relationships are exposed automatically: * `zendesk_ticket.requester` → `zendesk_user` (foreign key `requester_id`) * `zendesk_ticket.assignee` → `zendesk_user` (foreign key `assignee_id`) * `zendesk_ticket.organization` → `zendesk_organization` (foreign key `organization_id`) * `zendesk_user.organization` → `zendesk_organization` * `zendesk_user.requested_tickets` → `zendesk_ticket` * `zendesk_organization.users` → `zendesk_user` * `zendesk_organization.tickets` → `zendesk_ticket` * `ZendeskTicket.requester` → `ZendeskUser` (foreign key `requester_id`) * `ZendeskTicket.assignee` → `ZendeskUser` (foreign key `assignee_id`) * `ZendeskTicket.organization` → `ZendeskOrganization` (foreign key `organization_id`) * `ZendeskUser.organization` → `ZendeskOrganization` * `ZendeskUser.requested_tickets` → `ZendeskTicket` * `ZendeskOrganization.users` → `ZendeskUser` * `ZendeskOrganization.tickets` → `ZendeskTicket` ### Comments Zendesk has no standalone `/comments/{id}` endpoint, so comments are not exposed as their own collection. Instead, each ticket carries a structured `comments` array column that is fetched lazily from `/api/v2/tickets/{id}/comments` only when the projection asks for it (i.e. when `comments` is rendered on the detail view or referenced in a custom action). Each entry has the following shape: | Field | Type | Source | | -------------- | --------- | ------------------------------------------------------------------------------------------- | | `id` | `Number` | Zendesk comment id | | `body` | `String` | Plain-text body | | `html_body` | `String` | HTML-formatted body | | `public` | `Boolean` | `true` for public replies, `false` for internal notes | | `author_email` | `String` | Resolved through batched `users/show_many` calls (chunks of 100) across all visible authors | | `author_name` | `String` | Same | | `created_at` | `Date` | Comment creation timestamp | | Field | Type | Source | | -------------- | --------- | ------------------------------------------------------------------------------------------- | | `id` | `Number` | Zendesk comment id | | `body` | `String` | Plain-text body | | `html_body` | `String` | HTML-formatted body | | `public` | `Boolean` | `true` for public replies, `false` for internal notes | | `author_email` | `String` | Resolved through batched `users/show_many` calls (chunks of 100) across all visible authors | | `author_name` | `String` | Same | | `created_at` | `Date` | Comment creation timestamp | The column is read-only — comments are added by writing to the ticket's `description` on creation (Zendesk converts the description into the first comment). ### Custom fields Custom fields configured in your Zendesk account are introspected at boot and added to the matching collection's schema: * Ticket custom fields are exposed as `custom_` columns on `zendesk_ticket`. * User and organization custom fields are exposed using their Zendesk `key` (or `custom_` if no key is set) on `zendesk_user` / `zendesk_organization`. The Forest column type is derived from the Zendesk field type: | Zendesk field type | Forest column type | | ------------------------------------------------- | ------------------------------------------------- | | `text`, `textarea`, `regexp`, `partialcreditcard` | `String` | | `integer`, `decimal`, `lookup` | `Number` | | `date` | `Dateonly` | | `checkbox` | `Boolean` | | `dropdown`, `tagger` | `Enum` (or `String` if no options are configured) | | `multiselect` | `Json` | Unrecognized field types are skipped and logged with a warning; non-user-created (i.e. non-`removable`) ticket fields — which include every system ticket field — and inactive fields on any resource are dropped silently. Column-name collisions with a native column are also skipped with a warning. The initial introspection calls (`GET /ticket_fields.json`, `/user_fields.json`, `/organization_fields.json`) are not wrapped in a best-effort guard: a transport error during boot will fail `createZendeskDataSource` loudly. Make sure the API token's role has read access to the field definitions. * Ticket custom fields are exposed as `custom_` columns on `ZendeskTicket`. * User and organization custom fields are exposed using their Zendesk `key` (or `custom_` if no key is set) on `ZendeskUser` / `ZendeskOrganization`. The Forest column type is derived from the Zendesk field type: | Zendesk field type | Forest column type | | ------------------------------------------------- | ------------------ | | `text`, `textarea`, `regexp`, `partialcreditcard` | `String` | | `integer`, `decimal`, `lookup` | `Number` | | `date` | `Dateonly` | | `checkbox` | `Boolean` | | `dropdown`, `tagger` | `Enum` | | `multiselect` | `Json` | Inactive fields, system ticket fields (which already exist as native columns) and unrecognized types are skipped. If introspection fails (network error, missing scope, …) the datasource degrades gracefully: the corresponding collection is still registered without the custom fields, and a warning is logged. ## Capabilities ### Filters The condition tree is translated into a Zendesk Search API query. The following operators are supported per column type: | Column type | Supported operators | | ------------------ | --------------------------------------------------------------------------------- | | Primary key (`id`) | `Equal`, `In` | | `String`, `Enum` | `Equal`, `NotEqual`, `In`, `NotIn`, `Present`, `Blank` | | `Number` | `Equal`, `NotEqual`, `In`, `NotIn`, `Present`, `Blank`, `GreaterThan`, `LessThan` | | `Date`, `Dateonly` | `Equal`, `Before`, `After`, `Present`, `Blank` | | `Boolean` | `Equal`, `NotEqual` | Translations: | Operator | Zendesk Search syntax | | ---------------------- | ------------------------------------------------------------------- | | `Equal`, `NotEqual` | `field:value` / `-field:value` | | `In`, `NotIn` | repeated `field:value` clauses (Zendesk ANDs them — see note below) | | `GreaterThan`, `After` | `field>value` | | `LessThan`, `Before` | `field.json` (and friends); sibling conditions in the branch are then re-applied in memory. * **An empty `In` / `NotIn` raises `UnsupportedOperatorError`** rather than matching everything. * A handful of columns advertise no filter operators at all (so the UI offers no filter widget on them): `description`, `tags`, `url`, `comments` on tickets; `domain_names`, `details`, `notes`, `shared_tickets` on organizations; `time_zone`, `locale` on users. * A `null`/`undefined` value passed with `Equal` / `NotEqual` / `In` raises explicitly — use `Present` / `Blank` to filter for absence. * Filtering `zendesk_ticket.requester_email = "x@y.z"` is rewritten to Zendesk's `requester:x@y.z` operator. Only `Equal` is supported on `requester_email`. * String values containing whitespace, double quotes, parentheses, colons or hyphens are wrapped in double quotes (with embedded quotes escaped) before being sent to the Search API. The following operators are supported: | Operator | Translation | | ---------------------------------------------- | ------------------------------ | | `EQUAL`, `NOT_EQUAL` | `field:value` / `-field:value` | | `IN`, `NOT_IN` | repeated `field:value` clauses | | `GREATER_THAN`, `LESS_THAN`, `BEFORE`, `AFTER` | `field>value` / `field ### Sorting Only fields that the Zendesk Search API can sort on are honored. Other sort directives are silently ignored: * `zendesk_ticket`: `created_at`, `updated_at`, `priority`, `status`, `ticket_type` * `zendesk_user`: `created_at`, `updated_at`, `name` * `zendesk_organization`: `created_at`, `updated_at`, `name` * `ZendeskTicket`: `created_at`, `updated_at`, `priority`, `status`, `ticket_type` * `ZendeskUser`: `created_at`, `updated_at`, `name` * `ZendeskOrganization`: `created_at`, `updated_at`, `name` ### Pagination Forest's offset/limit pagination is translated to Zendesk's `page` / `per_page`. The Search API caps `per_page` at 100 (clamped automatically) **and caps the total result window at 1000 records** (`MAX_TOTAL_RESULTS`). A request with `skip + limit > 1000` raises `UnsupportedOperatorError` rather than silently returning a truncated set. A bulk `update` or `delete` that matches more than 1000 records will affect only the first 1000 and emit a `Warn` log — narrow the filter when working on larger sets. Forest's offset/limit pagination is translated to Zendesk's `page` / `per_page`. The Search API caps `per_page` at 100; larger limits are clamped. ### Aggregations Only `Count` aggregation without grouping is supported. Any other aggregation raises `UnsupportedOperatorError` — the Zendesk Search API has no group-by primitive. Count uses Zendesk's `/search/count.json` for filtered counts, and verifies record existence for the id-lookup path so it never over-counts. Only `Count` aggregation without grouping is supported. Any other aggregation raises a `ForestException`, the Zendesk Search API has no group-by primitive. ### Search The Forest search bar is shown by default on every collection (the agent's search decorator advertises `searchable: true`), but the default Forest search rewrites the user's query into an `Or` of `Contains` predicates — operators that the Zendesk Search translator does not support, so typing in the bar surfaces an `UnsupportedOperatorError`. You have two options: ```javascript theme={null} // Hide the bar agent.customizeCollection('zendesk_ticket', collection => collection.disableSearch(), ); // Or implement search yourself with a translator-compatible condition tree agent.customizeCollection('zendesk_ticket', collection => collection.replaceSearch(query => ({ field: 'subject', operator: 'Equal', value: query, })), ); ``` Filtering through the column filters keeps working as documented above either way. The free-text search bar is enabled on `ZendeskTicket`, `ZendeskUser` and `ZendeskOrganization`. The search term is appended to the query that is built from the active filters, so the count badge and the rendered list always agree. ### Writes Create, update and delete are supported on all three collections. Custom fields are folded into the appropriate Zendesk payload structure (`custom_fields` for tickets, `user_fields` / `organization_fields` for users and organizations). A few fields are intentionally read-only: * `id`, `url`, `created_at` and `updated_at` are ignored on writes. * On `zendesk_ticket`, `description` is only written on creation (Zendesk turns it into the first comment) — on update the value is dropped with a `Warn` log because Zendesk exposes no write endpoint for it. * `zendesk_ticket.requester_email` is computed at read time from the requester's profile and cannot be written directly. * The id-lookup short-circuit re-checks the caller's scopes/segments in memory, so a scoped `update` / `delete` never escapes its perimeter. * `id`, `url`, `created_at` and `updated_at` are ignored on writes. * On `ZendeskTicket`, `description` is only written on creation (where Zendesk turns it into the first comment); it is silently dropped on update because Zendesk exposes no write endpoint for it. * `ZendeskTicket.requester_email` is computed at read time from the requester's profile and cannot be written directly. ## Errors All exceptions raised by the datasource are subclasses of Forest's `BusinessError` / `ValidationError`: | Class | Parent | Raised by | | --------------------------- | ----------------- | -------------------------------------------------------------------------------------------------------------------- | | `ZendeskConfigurationError` | `ValidationError` | The client constructor when `subdomain` / `email` / `apiToken` is missing or empty. | | `ZendeskApiError` | `BusinessError` | Every HTTP call. Carries `operation`, HTTP `status` and the raw response `body`. | | `UnsupportedOperatorError` | `BusinessError` | The condition-tree translator (unsupported operator, `Or` aggregator, empty `In`/`NotIn`, …) and the pagination cap. | All three error classes are exported from `@forestadmin/datasource-zendesk` for use in your own catch blocks. Critical paths (search, count, ticket bulk fetch, ticket comment fetch, writes) raise a `ForestAdminDatasourceZendesk::APIError` that wraps the underlying Zendesk error. Configuration problems raise `ForestAdminDatasourceZendesk::ConfigurationError`. ## Logging Best-effort enrichment paths (bulk user/organization lookups, comment-author resolution, per-ticket comment fetches) log a `Warn` via the agent logger and degrade to a safe default (typically an empty `Map` or `null` fields) rather than failing the whole page render. Critical paths (search, count, ticket bulk fetch, ticket comment fetch, writes, custom-field introspection) raise `ZendeskApiError`. No retry / backoff is performed on Zendesk responses — if you need to absorb transient 429s or 502s, wrap your own retry layer around the `ZendeskHttpClient`. The datasource uses `Rails.logger` when available, and falls back to `Logger.new($stderr)`. You can override it explicitly: ```ruby theme={null} ForestAdminDatasourceZendesk.logger = MyLogger.new ``` Best-effort enrichment paths (bulk user/organization lookups, comment-author resolution, schema introspection) log a warning and degrade to a safe default rather than failing the whole page render. Critical paths (search, count, ticket bulk fetch, ticket comment fetch, writes) raise a `ForestAdminDatasourceZendesk::APIError` that wraps the underlying Zendesk error. ## Source code This connector is open source. Browse the code or contribute on GitHub: [`@forestadmin/datasource-zendesk`](https://github.com/ForestAdmin/agent-nodejs/tree/main/packages/datasource-zendesk) [`forest_admin_datasource_zendesk`](https://github.com/ForestAdmin/agent-ruby/tree/main/packages/forest_admin_datasource_zendesk) # Data Types Source: https://docs.forest.app/get-started/connect/data-types Reference of data types available in Forest Fields on Forest can either use `Primitive Types` or `Composite Types`. ## Primitive types The primitive types which are supported by Forest are the following: | Forest Type | Language Type | | ----------- | ---------------------------------------------------------- | | Boolean | Boolean | | Date | String with format "1985-10-26T01:22:00-08:00Z" (ISO-8601) | | Dateonly | String with format "1985-10-26" | | Enum | String | | JSON | Any JSON compatible value | | Number | Number | | Point | Array of 2 Numbers | | String | String | | Timeonly | String with format "01:22:00" | | Uuid | String with uuid v4 format | ## Composite types * Fields using composite types are not sortable and do not implement validation. * Fields that are an array of a primitive type **only** are filterable (depending on the data source). ```js theme={null} // Object containing 2 strings const typeOfObjectWithTwoStrings = { firstName: 'String', lastName: 'String' }; // Array of strings const typeOfArrayOfStrings = ['String']; // Array of objects const typeOfArrayOfObjects = [{ content: 'String' }]; // Object containing a 2d-grid of numbers const typeOfObjectContainingAGridOfNumbers = { content: [['Number']] }; ``` When using composite types, the data in the UI may not be displayed as you expect! | Composite Type | Example | How it gets displayed | | ------------------------------------- | ----------------------------------------------- | ------------------------------------------- | | Array of primitive type | `[ 'array', 'of', 'strings']` | As a custom widget in the edition form | | Object | `{ title: "the godfather"}` | As a nested form in the edition form | | Array of object | `[{ title: "the shawshank redemption"}]` | As a new collection in Related Data section | | Array of object (with nested objects) | `[{ rating: { kind: 'MPA", value: "PG-13" } }]` | JSON editor in the edition form | | Anything else | | JSON editor in the edition form | If you want to force displaying your data as a new Collection in the Related Data section, but can't because your data model contains nested objects, you may consider typing all nested objects as `'JSON'`. # Environment Variables Source: https://docs.forest.app/get-started/connect/environment-variables Configure your Forest back-end with required environment variables Environment variables are used to configure your Forest back-end securely, keeping sensitive information separate from your codebase. ## Required variables ### FOREST\_ENV\_SECRET Your unique environment secret provided by Forest. ```bash Node.js theme={null} FOREST_ENV_SECRET=1234567890abcdef1234567890abcdef1234567890abcdef ``` ```bash Ruby theme={null} FOREST_ENV_SECRET=1234567890abcdef1234567890abcdef1234567890abcdef ``` **Purpose:** * Authenticates your back-end with Forest * Links your back-end to your Forest project * Required for all architectures (Cloud, Self-Hosted, On-Premise) **Where to find it:** 1. Go to [app.forestadmin.com](https://app.forestadmin.com) 2. Select your project 3. Go to Settings → Environments 4. Copy the environment secret Never commit `FOREST_ENV_SECRET` to version control. Always use environment variables or secret management tools. ### FOREST\_AUTH\_SECRET Secret key used to sign authentication tokens (Self-Hosted and On-Premise only). ```bash Node.js theme={null} FOREST_AUTH_SECRET=your-secure-random-string-at-least-32-characters-long ``` ```bash Ruby theme={null} FOREST_AUTH_SECRET=your-secure-random-string-at-least-32-characters-long ``` **Purpose:** * Signs JWT tokens for user authentication * Required for Self-Hosted and On-Premise architectures * Not needed for Cloud architecture **Generate a secure secret:** ```bash theme={null} # Generate a random 32-character string node -e "console.log(require('crypto').randomBytes(32).toString('hex'))" # Or use OpenSSL openssl rand -hex 32 ``` ### NODE\_ENV (Node.js only) Environment mode for Node.js applications. ```bash theme={null} NODE_ENV=production # Options: development, production, test ``` **Purpose:** * `production`: Optimized performance, minimal logging * `development`: Detailed logging, development mode * `test`: Testing mode ```javascript Node.js theme={null} const agent = createAgent({ envSecret: process.env.FOREST_ENV_SECRET, authSecret: process.env.FOREST_AUTH_SECRET, isProduction: process.env.NODE_ENV === 'production' }); ``` ```ruby Ruby theme={null} # config/initializers/forest_admin_rails.rb ForestAdminRails.configure do |config| config.env_secret = ENV['FOREST_ENV_SECRET'] config.auth_secret = ENV['FOREST_AUTH_SECRET'] end ``` **Security reminder:** These environment variables contain sensitive secrets that authenticate your back-end with Forest. Never commit them to version control, share them publicly, or reuse them across different environments (development, staging, production). Always use separate secrets for each environment and store them securely using environment variables or secret management tools. # MCP Connectors Source: https://docs.forest.app/get-started/connect/integrations/mcp-servers Connect external tools to Forest using the Model Context Protocol: native connectors and custom servers. **MCP (Model Context Protocol)** is a standardized way to connect external tools, APIs, and data sources to Forest. Forest ships **native connectors** for popular tools, and lets you plug in **any custom MCP server**. MCP connectors are used inside workflows to trigger external actions, fetch data from external systems, and build multi-step processes that span several tools. ## What is MCP? The Model Context Protocol is an open standard for connecting AI assistants and applications to external tools and data sources. In Forest, an MCP connector exposes a third-party tool's **tools** (its callable operations) so you can use them in your workflows. Each native connector wraps the tool's **official MCP server**. Forest handles the connection and authentication, and the operations available are the ones that vendor's MCP server exposes. For the exhaustive list of operations a connector provides, follow the vendor reference linked in each connector below. ## Native MCP Connectors Connect any of these from **Project Settings → Integrations → Add MCP Server**, then pick the connector instead of entering a URL. ### Payments & billing Payments, customers, subscriptions, refunds. Payments, orders, catalog, customers. ### CRM & support Contacts, companies, deals, tickets. Conversations, contacts, articles. Tickets, users, organizations. ### Productivity & docs Mail, calendar, files, Teams. Pages, databases, search. Boards, items, updates. ### Communication Messages, channels, users. ### Analytics & data Events, charts, cohorts. Query tables and views. ### Automation Trigger and run n8n workflows. Trigger Zaps. Trigger recipes. ### Other Kolar operations. Orias registry lookups. The exact operations available come from each vendor's MCP server and can change without notice, which is why Forest doesn't duplicate the full tool list here. After connecting, the **"Confirm the exposed tools"** step shows you exactly what's available for your account and plan. ### Example: Stripe Each native connector follows the same flow. For Stripe: 1. Go to **Project Settings → Integrations → Add MCP Server**. 2. Select **Stripe**. 3. Complete the connector's authentication. 4. Confirm the exposed tools, then save. **What it unlocks**: read and act on customers, charges, subscriptions, invoices and refunds from your workflows. For the full operation reference, see the [Stripe MCP documentation](https://docs.stripe.com/mcp). ## Connect a custom MCP server If a tool isn't in the list above, connect any MCP server by URL: 1. Navigate to **Project Settings → Integrations**. 2. Click **Add MCP Server**. 3. Enter your MCP server URL and configuration: ```json theme={null} { "url": "https://your-mcp-server.com", "name": "My Custom MCP Server", "apiKey": "your-api-key" } ``` 4. Check that all the expected tools are exposed. 5. Save your configuration. Learn more about building and hosting MCP servers in the [Claude MCP documentation](https://docs.claude.com/en/docs/agents-and-tools/remote-mcp-servers). # All integrations Source: https://docs.forest.app/get-started/connect/integrations/overview Every third-party tool Forest connects to: data sources, action plugins, embeds, and AI assistants. Forest integrates with third-party tools in several ways: as a **data source** (browse and edit the tool's data inside Forest), as an **action plugin** (trigger the tool from a Forest action), as an **embed** (surface Forest inside another product), as an **MCP connector** (call the tool from a Forest workflow), or as an **AI assistant** connection. A tool can offer more than one path. See the [MCP Connectors](/get-started/connect/integrations/mcp-servers) catalog for the full list of workflow connectors. This page lists everything Forest connects to today and where to find it. Looking for a tool that isn't here? [Contact us](mailto:support@forestadmin.com), most APIs can be connected with a [custom data source](/get-started/connect/data-sources/custom-datasources/overview). ## Support & ticketing * **Browse & edit** tickets, users and organizations: [Zendesk data source](/get-started/connect/data-sources/zendesk) * **Create & close tickets** from a Forest action: [Zendesk plugins](/product/process/advanced-concepts/plugins/zendesk) * **Embed Forest** inside Zendesk tickets: [Zendesk app](/product/embed/zendesk) * **Call Zendesk from workflows**: [Zendesk MCP Connector](/get-started/connect/integrations/mcp-servers) ## Payments & billing * **Browse & edit** customers, charges and subscriptions as collections: [Stripe data source](/get-started/connect/data-sources/stripe) * **Call Stripe from workflows**: [Stripe MCP Connector](/get-started/connect/integrations/mcp-servers) ## CRM & marketing * **Browse & edit** objects (contacts, companies, deals) as collections: [HubSpot data source](/get-started/connect/data-sources/hubspot) * **Call HubSpot from workflows**: [HubSpot MCP Connector](/get-started/connect/integrations/mcp-servers) ## Data, search & storage Import tables and views (read-only, via ODBC) as a [data source](/get-started/connect/data-sources/snowflake), or query Snowflake from workflows with the [MCP Connector](/get-started/connect/integrations/mcp-servers). Expose Elasticsearch indices as searchable collections. Manage file attachments backed by S3 (and other backends) with the Active Storage plugin, or the AWS S3 plugin (see [Plugins](/product/process/advanced-concepts/plugins/overview)). Connect Airtable bases as collections. Connect a Cosmos DB account as collections. Connect a GraphQL or Hasura endpoint as collections. ## Automation Read and act on Forest data from n8n with the [Forest node](/product/embed/n8n), or trigger n8n workflows from Forest with the [MCP Connector](/get-started/connect/integrations/mcp-servers). ## AI assistants Expose your data and actions to AI assistants over the Model Context Protocol. Route workflow AI steps through your own OpenAI or Anthropic keys with Forest Runtime. ## Databases & ORMs Forest connects natively to most SQL and NoSQL databases, as well as to your existing ORM models (Sequelize, Mongoose, Active Record, Mongoid). See the [data sources overview](/get-started/connect/data-sources/overview) for the full list. ## Build your own Need a tool that isn't listed here? Wrap any REST or internal API as a Forest collection with a [custom data source](/get-started/connect/data-sources/custom-datasources/overview), or factor a repeated customization into a [plugin](/product/process/advanced-concepts/plugins/overview). # Overview Source: https://docs.forest.app/get-started/connect/overview Load your data into Forest and enrich it with additional sources Forest connects to your databases and external services to expose your data and make it actionable. ## How data flows into Forest Forest loads data through **datasources**, connections to your databases, APIs, or any system that holds data. ```javascript Node.js theme={null} const agent = createAgent(options); // Connect your primary database agent.addDataSource( createSqlDataSource('postgresql://localhost/mydb') ); // Add a second datasource (e.g. MongoDB) agent.addDataSource( createMongooseDataSource(mongoConnection) ); // Forest automatically: // - Discovers all tables // - Detects column types // - Identifies relationships (foreign keys) // - Creates collections for each table ``` ```ruby Ruby theme={null} agent = Forestadmin::Agent.new(options) # Connect your primary database agent.add_datasource( Forestadmin::Datasource::ActiveRecord.new ) # Add a second datasource agent.add_datasource( Forestadmin::Datasource::Mongoid.new ) # Forest automatically: # - Discovers all tables # - Detects column types # - Identifies relationships (foreign keys) # - Creates collections for each table ``` ## Forest datasources Forest loads data through **datasources**, connections to your databases, APIs, or any system that holds data. For some datasources, Forest performs introspection to discover the schema automatically. For others (like ORM-based connections), the schema is already known from your model definitions. **ORM-based connections** (Sequelize, Mongoose): * Forest reads your ORM model definitions * Uses the schemas you've already defined in your application code * Detects relationships from model associations * No database introspection needed - everything comes from your models **Direct database connections** (SQL, MongoDB): * **SQL databases**: Forest queries the database metadata (information\_schema or system catalogs) to discover tables, columns, data types, constraints, foreign keys, and indexes * **MongoDB**: Forest samples a subset of documents (default: 100 per collection) and analyzes them to infer the schema structure, including nested fields and references **API-based connections** (REST, GraphQL): Forest uses the API schema or a custom mapping to expose data. Once introspection is complete, Forest automatically: 1. **Discovers schema** - all tables/collections and their structure 2. **Maps data types** - converts database types to Forest types 3. **Detects relationships** - foreign keys or references become navigable relationships 4. **Creates collections** - each table/collection becomes a collection in the UI 5. **Enables CRUD operations** - browse, create, edit, delete records ## Loading more data: multi-datasources Connect multiple databases or APIs in the same back-end: ```javascript Node.js theme={null} // Primary PostgreSQL database agent.addDataSource( createSqlDataSource('postgresql://localhost/main'), { name: 'main' } ); // MongoDB for analytics agent.addDataSource( createMongooseDataSource(mongoConnection), { name: 'analytics' } ); ``` ```ruby Ruby theme={null} # Primary PostgreSQL database agent.add_datasource( Forestadmin::Datasource::ActiveRecord.new, name: 'main' ) # MongoDB for analytics agent.add_datasource( Forestadmin::Datasource::Mongoid.new, name: 'analytics' ) ``` **Looking to enrich a single record?** If you want to add computed fields or fetch external data to enrich individual records (not load entire collections), use **Smart Fields** instead of adding a datasource. [Learn how to enrich data with computed fields →](/product/process/fields/computed) [Browse available datasources →](/get-started/connect/data-sources/overview) # Relationships Source: https://docs.forest.app/get-started/connect/relationships-schema Define and understand relationships between collections in your Forest schema A join is used to combine rows from two or more tables, based on a related column between them. ## Declaration In Forest, relations are defined as fields and are traversable in only one direction. ## Join types Four join types are available: `ManyToOne`, `ManyToMany`, `OneToMany`, and `OneToOne`. | Type | Where are the common keys? | | ---------- | -------------------------------------------------------------------------------------------------- | | ManyToOne | `origin[foreignKey] == foreign[foreignKeyTarget]` | | OneToMany | `origin[originKeyTarget] == foreign[originKey]` | | ManyToMany | `origin[originKeyTarget] == through[originKey] && though[foreignKey] == foreign[foreignKeyTarget]` | | OneToOne | `origin[originKeyTarget] == foreign[originKey]` | # TypeScript Autocompletion Source: https://docs.forest.app/get-started/connect/typescript-autocompletion Enable TypeScript autocompletion and type safety for your Forest Node.js back-end **Node.js only** - This page covers TypeScript autocompletion setup for the Node.js agent. For information about Forest data types, see [Data Types](/get-started/connect/data-types). The Forest Node.js back-end, built entirely in TypeScript, provides comprehensive autocompletion capabilities for collection names, field names, and handler parameters. ## Generating a Typing File The back-end can generate a typing file based on your data models. This file is auto-generated and should not be manually edited. ### Configuration Options Two configuration options control typing file generation: * **`typingsPath`**: Specifies the location where the typing file will be created * **`typingsMaxDepth`**: Controls the maximum introspection depth for relationships ## TypeScript Usage In TypeScript projects, import and template the generated schema: ```typescript theme={null} import { createAgent } from '@forestadmin/agent'; import { Schema } from './typings'; import transactions from './customization/transactions'; await createAgent({ // ... typingsPath: './typings.ts', typingsMaxDepth: 5, }) .customizeCollection('transactions', transactions) .mountOnStandaloneServer(3000) .start(); ``` The `customizeCollection` method and handler parameters receive strong typing through the Schema template. ## JavaScript Usage JavaScript developers can leverage JSDoc syntax to maintain autocomplete capabilities: ### Main Back-end File ```javascript theme={null} const { createAgent } = require('@forestadmin/agent'); /** * @typedef {import('@forestadmin/agent').Agent} Agent * @typedef {import('../typings').Schema} Schema */ /** * @type {Agent} */ const agent = createAgent({ typingsPath: './typings.ts', typingsMaxDepth: 5, }); ``` ### Customization Files In separate customization files, JSDoc type annotations preserve autocompletion: ```javascript theme={null} /** * @param {CollectionCustomizer} transactions */ module.exports = transactions => { transactions.removeField('amountInEur'); }; ``` # Audit & Activity Logs Source: https://docs.forest.app/get-started/control/audit Track and monitor all user actions in Forest ## Overview The Activity tab provides a comprehensive view of team activities within Forest. Every action performed by users is tracked and logged with complete context for accountability and compliance. ## What gets logged Forest tracks all user activities across different areas: | Category | Activities | | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Data Operations** | `create` - Record creation
`update` - Record updates
`delete` - Record deletion | | **Actions** | `action` - Action execution
`createApproval` - Approval request created
`approveApproval` - Approval granted
`rejectApproval` - Approval rejected
`markApprovalAsFailed` - Approval failed
`ignoreApprovalWarning` - Warning bypassed
`sendApprovalBackInReview` - Sent back for review | | **Inbox Management** | `startProcessingInboxTask` - Task processing started
`assignUserToInboxTask` - Task assigned
`unassignUserFromInboxTask` - Task unassigned
`cancelInboxTask` - Task cancelled
`autoCancelInboxTask` - Task auto-cancelled | | **Workflows** | `triggerWorkflow` - Workflow initiated
`completeWorkflowStep` - Step completed
`selectWorkflowOption` - Option selected
`reviseWorkflowStep` - Step revised
`resumeWorkflow` - Workflow resumed
`completeWorkflow` - Workflow finished
`abortWorkflow` - Workflow aborted | | **Collaboration** | `createNote` - Note added
`deleteNote` - Note removed
`createMessage` - Message sent | For each action, the system logs the **user, targeted record(s), and timestamp**. Forest tracks and stores these activities **without records' sensitive data** except the ID/primary key. **Known Limitation**: When collections share identical names with different capitalization (e.g., `myCollection` and `MyCollection`), activity tracking may incorrectly attribute actions between these collections. ## Viewing activity logs ### Global activity tab Access the Activity tab to see all team activities across your Forest project. The logs show all actions with user information, affected records, and timestamps. ### Record-level activity Individual records display their own activity history through a dedicated Activity tab on each record's detail page, enabling team members to track specific record changes over time. ## Export all activities Admin users can export comprehensive activity data including read operations: * Search activities * Filter usage * Record detail views * Dashboard access * Record exports * All write operations listed above To export activity logs: 1. Click the "Export" button in the Activity tab 2. Select a date range 3. Initiate the export You will receive download links via email shortly after the export request. **API Access**: Activity logs are also available programmatically through the Forest API. See the [Activity Logs API reference](/reference/api/endpoints/activity-logs) for more information. ## Admin Logs While Activity Logs track what operators do with your **data**, **Admin Logs** track changes to your Forest project's **configuration**, such as permission and access-management changes. They are kept separately from the data activity logs. Admin Logs are available through the Forest public API, which lists the admin logs for a project, most recent first. This endpoint is not self-service: [contact us](https://www.forestadmin.com/contact) to enable access. See the [Admin Logs API reference](/reference/api/endpoints/admin-logs) for details. # Two-Factor Authentication (2FA) Source: https://docs.forest.app/get-started/control/authentication/2fa Enable and enforce two-factor authentication to add an extra layer of security to user accounts ## Overview Two-Factor Authentication (2FA) adds an additional security layer beyond username and password. Users must provide a second factor - typically a time-based one-time password (TOTP) from an authenticator app - to access Forest. **Recommended for All Users**: 2FA significantly reduces the risk of unauthorized access, even if passwords are compromised. ## Supported 2FA methods ### Authenticator apps (recommended) Time-based One-Time Password (TOTP) apps generate 6-digit codes that change every 30 seconds: Free app for iOS and Android Supports backup and cloud sync Multi-device support with encrypted backups Password manager with built-in TOTP Open-source with TOTP support One-tap push notifications ### Backup codes Recovery codes to use if you lose access to your authenticator app: * Generated during 2FA setup * One-time use only * Store securely (password manager or printed copy) * Can regenerate if needed **Save Your Backup Codes**: Without backup codes or access to your authenticator, you'll be locked out if your device is lost. ## Enabling 2FA (for users) ### Setup process Click your profile picture > **Account Settings** > **Security** Click **Enable 2FA** button 1. Open your authenticator app 2. Tap "Add account" or "+" button 3. Scan the QR code displayed in Forest **Or** enter the setup key manually if you can't scan Enter the 6-digit code from your authenticator app to confirm it's working correctly 1. Download or copy your backup codes 2. Store them securely (password manager recommended) 3. Check the box to confirm you've saved them You'll now be prompted for a code each time you log in ### Manual setup key If you can't scan the QR code, use the manual setup key: ``` Setup Key: JBSWY3DPEHPK3PXP Account: your-email@example.com Type: Time-based ``` ## Using 2FA to log in ### Login flow Enter your email and password as usual Open your authenticator app and enter the current 6-digit code Check "Trust this device for 30 days" to skip 2FA on this device You're logged in to Forest ### Using backup codes If you don't have access to your authenticator app: On the 2FA prompt, click the link to use a backup code Enter one of your saved backup codes (case-insensitive) Each backup code can only be used once. Generate new ones if running low. ### Trusted devices Mark devices as trusted to skip 2FA for 30 days: * **Use Case**: Your primary work computer * **Security**: A secure cookie identifies the device * **Removal**: Go to Account Settings > Security > Trusted Devices to revoke **Public Computers**: Never mark public or shared computers as trusted. ## Enforcing 2FA (for admins) Administrators can require 2FA for all users or specific roles. ### Project-wide enforcement Require 2FA for everyone: Go to **Project Settings** > **Security** > **Authentication** Toggle **Require Two-Factor Authentication** to ON Choose how long users have to enable 2FA: * 24 hours (urgent) * 7 days (recommended) * 30 days (gradual rollout) Forest automatically emails users about the requirement Track which users have enabled 2FA in **Project Settings** > **Teams** > **Users** ### Role-based enforcement Require 2FA only for specific roles: ```javascript theme={null} // Example: Require 2FA for Admins and Editors { "roles": { "admin": { "require2FA": true }, "editor": { "require2FA": true }, "viewer": { "require2FA": false // Optional for viewers } } } ``` **Configuration**: 1. Go to **Project Settings** > **Roles** 2. Edit each role 3. Check **Require 2FA for this role** ### Exceptions Allow specific users to bypass 2FA requirement: * **Use Case**: Emergency access accounts, service accounts, external contractors * **Configuration**: Edit user profile > **Security** > **Exempt from 2FA requirement** **Best Practice**: Minimize exceptions. If a user truly needs access, they should enable 2FA. ## Managing 2FA ### Regenerating backup codes If you've used all your backup codes or lost them: Go to **Account Settings** > **Security** Click **Regenerate Backup Codes** Verify with current authenticator code Old codes are invalidated. Save the new ones securely. ### Resetting your own 2FA If you need to switch authenticator apps or devices: Go to **Account Settings** > **Security** > **Disable 2FA** Enter your current 2FA code or a backup code Follow the setup process again with your new device/app ### Admin: resetting user's 2FA If a user loses access to their authenticator and backup codes: **Security Risk**: Only reset 2FA after verifying the user's identity through alternate means (video call, ID verification, etc.). Confirm the user's identity (don't rely solely on email, which could be compromised) Go to **Project Settings** > **Teams** > **Users** Search for the user who needs 2FA reset Click user menu (...) > **Reset Two-Factor Authentication** Confirm the reset. User must set up 2FA again at next login (if required). This action is automatically logged in audit logs for security tracking ## 2FA + SSO ### How they work together 2FA and SSO can be used simultaneously for defense in depth: **MFA enforced at identity provider** * User authenticates with IdP (e.g., Okta, Azure AD) * IdP requires MFA (push notification, TOTP, etc.) * Forest trusts the IdP's authentication **Advantages**: * Centralized MFA management * One MFA prompt for all applications * Better user experience **Configuration**: Enable MFA in your IdP settings **Additional 2FA layer in Forest** * User authenticates via SSO * Forest requires its own 2FA **Advantages**: * Extra layer of security * Works even if IdP MFA is disabled * Forest-specific second factor **Use Case**: Very high security environments, compliance requirements **Rely entirely on IdP security** * User authenticates via SSO * No additional 2FA in Forest * IdP should enforce MFA **Advantages**: * Simplified user experience * Consistent with other applications **Requirement**: IdP must have strong MFA policies ### Recommended configuration Enforce MFA at the IdP level Disable Forest 2FA for SSO users to avoid double-prompting Require Forest 2FA These users don't benefit from IdP security ## Troubleshooting ### Code not working **Problem**: Authenticator app time is out of sync **Symptoms**: Code is always rejected, even when entered correctly **Solution**: 1. Check your phone's time settings 2. Enable automatic time/date 3. Try the next code (they change every 30 seconds) **For Google Authenticator**: * Go to Settings > Time correction for codes > Sync now **Problem**: Entering an old or incorrect code **Solution**: * Wait for the code to refresh in your app * Ensure you're using the correct account (if you have multiple) * Check for typos (0 vs O, 1 vs l) **Problem**: Removed app or factory reset phone without backing up **Solution**: * Use a backup code if you have one * Contact your admin for 2FA reset * Admin must verify your identity before resetting ### Can't scan QR code **Solutions**: 1. **Use Manual Entry**: Copy the setup key and enter it manually in your authenticator app 2. **Try Different Device**: Use a tablet or another phone to scan 3. **Check Camera Permissions**: Ensure authenticator app has camera access 4. **Screenshot**: Take a screenshot (secure it afterwards) and scan from photos ### Lost backup codes **If you still have authenticator access**: 1. Log in with your authenticator code 2. Regenerate new backup codes 3. Save them securely **If you don't have authenticator or backup codes**: 1. Contact your administrator 2. Admin will verify your identity 3. Admin can reset your 2FA 4. Set up 2FA again immediately ### Can't log in after 2FA enforcement **Problem**: 2FA was enforced but user hasn't set it up **Solution**: 1. Users receive grace period to enable 2FA 2. During grace period, they're prompted to set up 2FA 3. After grace period, they must set up 2FA before accessing **Admin Override**: * Admin can temporarily exempt user from 2FA requirement * User can then log in and set up 2FA properly # SCIM Integration with Okta Source: https://docs.forest.app/get-started/control/authentication/scim-okta Automate user management by integrating Okta's SCIM provisioning with Forest You need administrator access to the Forest project. ## Supported Features The Okta SCIM integration enables: * User provisioning from Okta to Forest * Updating user roles, permission levels, and tags * Deleting users when removed from the Forest app in Okta * SCIM Groups for team assignment * Read-only `userName` (email format) and name fields post-creation ## Setup Process ### Step 1: Add Forest App Navigate to Okta Applications tab, browse the app catalog, and select Forest. Assign a descriptive label. ### Step 2: Authenticate Okta Enable the User provisioning feature in Forest project settings. This generates an API token to paste into Okta's Integration tab. ### Step 3: Configure Mapping Rules Create rules for mandatory fields: `teams`, `role`, `permissionLevel`, and optional `tags`. Values must match existing Forest configurations. Ensure mapping direction flows from Okta to Forest. **Required Parameters:** * `permissionLevel`: Must be `admin`, `editor`, `user`, or `developer` * `role`: Must match existing Forest roles * `teams`: Team names for user assignment * `tags`: Optional key/value pairs for user tagging ### Step 4: Manage Groups In Okta's Directory section, create groups matching Forest teams. Use the "Push groups" tab to link Okta groups with Forest teams. Optionally disable group renaming to prevent Okta from overwriting team names. Removing a group in Okta that was created from, or linked to, a Forest team will **delete** that Forest team. When you link an Okta group to a Forest team, the team is renamed to match the group name, unless you disable that option. ## Custom Attributes Add custom user attributes via Directory > Profile Editor for enhanced mapping flexibility. ## Troubleshooting * Verify `permissionLevel` values: admin, editor, user, or developer * Confirm `role` matches existing Forest roles * Allow time for synchronization * Note that team updates may trigger back-end restarts # Manual SCIM Integration with Okta Source: https://docs.forest.app/get-started/control/authentication/scim-okta-manual Manually configure Okta's SCIM provisioning with Forest using the SCIMForest 2.0 Test App Enabling SCIM disables Forest user editing. All user management must be done through Okta. ## Supported Features The manual Okta SCIM integration enables: * User provisioning from Okta to Forest * Updating user role, permission level, and tags * User deletion when removed from the Okta app * Team assignment via groups ## Setup Steps ### 1. Add Forest App Navigate to Applications > Browse App Catalog, then select "SCIMForest 2.0 Test App (Header Auth)". Name the application, keeping in mind each app links to one Forest project. ### 2. Authentication Generate a provisioning token in Forest project settings. In Okta's Integration tab, enter the token prefixed with "Bearer" (format: "Bearer \[token]"). ### 3. Configuration Keep "Sync Password" disabled as it's unsupported. ### 4. Custom Parameters Four parameters require configuration: * `permissionLevel`: Admin, Developer, Editor, or User * `teams`: comma-separated team names (e.g., "Operators,Support") * `role`: must match existing project roles * `tags`: optional key/value pairs separated by semicolons ### 5. Attribute Setup In Profile Editor, set external namespace to `urn:ietf:params:scim:schemas:extension:forest:2.0:User` ### 6. Mapping Rules Create rules directing Okta to Forest for automatic `role`, `permissionLevel`, and `tags` assignment. ### 7. Group Management Configure Directory groups for team mapping, then use "Push Groups" to link Okta groups with Forest teams. Optional: disable automatic team renaming in app settings. # SCIM Integration with OneLogin Source: https://docs.forest.app/get-started/control/authentication/scim-onelogin Automate user management by integrating OneLogin's SCIM provisioning with Forest Enabling SCIM disables Forest user editing. All user management must be done through OneLogin. ## Supported Features The OneLogin SCIM integration enables: * User provisioning from OneLogin to Forest * Updating user role, permission level, and tags * Deleting users when removed from the Forest app in OneLogin * SCIM Groups for team assignment ## Configuration Steps ### 1. Adding the Forest App Navigate to OneLogin's Application tab, select "Add App," then search for and select "SCIM Provisioner with SAML (SCIM v2 Core)." ### 2. Authentication Setup Name your app, then enable User provisioning in Forest project settings. This generates a token to paste into OneLogin. ### 3. SCIM Base URL Add this endpoint: `https://api.forestadmin.com/scim` ### 4. JSON Template Configuration The SCIM template includes user schemas with custom Forest parameters for permissionLevel, role, tags, and teams. ### 5. Custom Parameters * **permissionLevel**: Must match existing Forest permission level exactly * **role**: Must match existing project role exactly * **teams**: Comma-separated team names (e.g., "Operators,Support") * **tags**: Key/value pairs separated by semicolons (e.g., "regions:France,Italie;job:developer") ### 6. Mapping Rules Create rules to automatically provide mandatory parameters (role, permissionLevel) and optional tags. ### 7. Custom User Attributes Add custom fields in the Users tab under "Custom User Fields" to base mapping rules on. ### 8. SCIM Groups Management Refresh entitlements to fetch OneLogin roles, then create mapping rules between OneLogin roles and Forest teams. # Single Sign-On (SSO) Source: https://docs.forest.app/get-started/control/authentication/sso Authenticate users through your identity provider using SAML 2.0 Single Sign-On lets your users access Forest with their existing corporate credentials, managed by your identity provider (IdP). Forest supports the **SAML 2.0** specification, so any SAML 2.0-compliant IdP can be used. SSO is configured once, at the **organization** level, by an **organization owner**. A single identity provider is configured per organization. ## How SSO works Forest uses a standard SP-initiated SAML 2.0 flow: the user starts at Forest, authenticates against your IdP, and is redirected back with a signed assertion. SP-initiated SAML 2.0 flow: the user starts at Forest, authenticates against the identity provider, and is redirected back with a signed assertion ## Forest SAML settings When you declare Forest as an application in your IdP, use these values (the audience / Entity ID is shown in your organization settings): | Setting | Value | | ------------------ | -------------------------------------------------------------- | | Callback / ACS URL | `https://api.forestadmin.com/api/saml/callback` | | Sign-on URL | `https://api.forestadmin.com/api/saml/callback` | | Logout URL | `https://app.forestadmin.com/login` | | `NameID` | The user's **email address** (must match their Forest account) | ## Configuring SSO As an organization owner, go to **Organization settings → Security** and open the SSO configuration. Create a SAML 2.0 application in your IdP using the Forest SAML settings above. Make sure the `NameID` it returns is the user's email address. Give Forest your IdP's metadata in one of these ways: * **XML metadata endpoint URL** (recommended): paste the metadata URL exposed by your IdP. * **XML metadata file**: upload the metadata file downloaded from your IdP. * **Manual entry**: enter the login endpoint, the logout endpoint, and a valid signing certificate. Test the configuration, then enable it. Once SSO is enabled, all users must log in again. Users must already exist in Forest (or be provisioned through [SCIM](/get-started/control/authentication/scim-okta)) with the same email address used by your IdP. The `NameID` returned in the SAML assertion must equal that email. ## How users log in with SSO On the Forest login page, click **"Login with SSO"**, enter your organization name, and click **"Login"**. The user is redirected to your IdP and back to Forest once authenticated. ## IdP-initiated login (optional) Forest also accepts IdP-initiated logins, where the user starts from your IdP's portal and opens Forest from there. IdP-initiated login introduces a security risk associated with CSRF in the SAML protocol. Prefer SP-initiated login (starting from Forest) unless you specifically need the IdP-initiated flow. ## Provider guides Forest works with any SAML 2.0 identity provider. Step-by-step guides are available for the most common ones: # Azure AD / Entra ID SSO Source: https://docs.forest.app/get-started/control/authentication/sso-providers/azure Configure SSO with Azure Active Directory / Microsoft Entra ID ## Configuration 1. In the Azure Active Directory admin center, go to **Enterprise applications → New application** 2. Select **Create your own application** (Forest is not in the gallery) 3. Choose **Integrate any other application you don't find in the gallery (Non-gallery)** 4. Configure SAML settings: | Setting | Value | | ---------------------- | ----------------------------------------------------------------------------------------- | | Reply URL (ACS URL) | `https://api.forestadmin.com/api/saml/callback` | | Sign on URL | `https://api.forestadmin.com/api/saml/callback` | | Identifier (Entity ID) | `forestadmin-YourOrganizationName` | | Logout URL (optional) | `https://app.forestadmin.com/login` | | Relay State (optional) | `{"organizationName": "YourOrganizationName", "destinationUrl": "organization.projects"}` | 5. In the **SAML Signing Certificate** section, copy the **App Federation Metadata Url** 6. In Forest Organization settings, select **XML file endpoint** and paste the URL ## Troubleshooting * Double-check all endpoints and certificate expiration dates * Ensure `nameID` is configured to use the **email address used on Forest accounts** # Generic SAML 2.0 SSO Source: https://docs.forest.app/get-started/control/authentication/sso-providers/generic-saml Configure SSO with any SAML 2.0-compatible Identity Provider Forest supports any Identity Provider that implements the SAML 2.0 specification. You must be an Organization Owner. ## Step 1: Configure your Identity Provider Declare Forest as a Service Provider in your IdP using these values: | Setting | Value | | ---------------------- | ----------------------------------------------- | | Callback URL / ACS URL | `https://api.forestadmin.com/api/saml/callback` | | Sign on URL | `https://api.forestadmin.com/api/saml/callback` | | Logout URL | `https://app.forestadmin.com/login` | | Audience (EntityID) | Displayed in your Forest Organization settings | ## Step 2: Configure Forest In your Organization settings → **Security** tab, configure Forest with your IdP's information. **Option 1: XML metadata endpoint (recommended)** Provide the URL to your IdP's metadata XML endpoint. This supports automatic certificate rotation without service interruption. **Option 2: XML file upload** Upload the metadata XML file generated by your IdP. **Option 3: Manual input** Enter manually: * Login endpoint * Logout endpoint * Valid certificate ## Step 3: Test and enable Click **"Test configuration"** to verify authentication works. Once confirmed, enable SSO for all users. After enabling SSO, all users will be required to log in again. ## IdP-initiated login (optional) Enable **IdP-initiated login** to allow users to be redirected to Forest directly from your IdP dashboard. Set this Relay State on your IdP: ```json theme={null} { "organizationName": "YourOrganizationName", "destinationUrl": "organization.projects" } ``` ## Troubleshooting * Double-check all endpoints and certificate expiration dates * Ensure the `NameID` in your IdP is set to the **email address used on Forest accounts** * Ensure your IdP is configured for **SAML 2.0** If you can't resolve the issue, ask for help on the [Forest Community Forum](https://community.forestadmin.com). # Google Workspace SSO Source: https://docs.forest.app/get-started/control/authentication/sso-providers/google Configure SSO with Google Workspace ## Configuration 1. Log in to your Google account and navigate to the **Admin console** 2. Go to **Menu → Apps → Web and mobile apps** 3. Click **Add App → Add custom SAML app** and follow the wizard 4. In the **Service Provider Details** window, enter: | Setting | Value | | -------------------- | ------------------------------------------------------------------------------------------------------------------ | | ACS URL | `https://api.forestadmin.com/api/saml/callback` | | Entity ID | Displayed in your Forest Organization settings | | Start URL (optional) | For [IdP-initiated login](/get-started/control/authentication/sso-providers/overview#idp-initiated-login-optional) | 5. Download or copy the IdP metadata and paste it into Forest (see [SSO configuration guide](/get-started/control/authentication/sso-providers/overview)) See the [Google documentation on custom SAML applications](https://support.google.com/a/answer/6087519) for more details. ## Troubleshooting * Double-check all endpoints and certificate expiration dates * Ensure the `Name ID` (primary email) in your IdP matches the **email address used on Forest accounts** # Okta SSO Source: https://docs.forest.app/get-started/control/authentication/sso-providers/okta Configure SSO with Okta ## Configuration 1. In your Okta admin dashboard, click **Create a new app integration** 2. Select **SAML 2.0** and follow the wizard 3. Configure the app with these settings: | Setting | Value | | ------------------------------ | ----------------------------------------------------------------------------------------- | | ACS URL | `https://api.forestadmin.com/api/saml/callback` | | Audience URI (EntityID) | `forestadmin-YourOrganizationName` | | Name ID format | **EmailAddress** | | Application username | **Email** | | Update application username on | **Create and update** | | Relay State (optional) | `{"organizationName": "YourOrganizationName", "destinationUrl": "organization.projects"}` | 4. Go to the **Sign On** tab → **Metadata details** and copy the **Metadata URL** 5. In Forest Organization settings, select **XML file endpoint** and paste the Metadata URL ## Troubleshooting * Double-check all endpoints and certificate expiration dates * Ensure `Name ID format` is set to **EmailAddress** and matches the email used on Forest accounts # SSO Provider Guides Source: https://docs.forest.app/get-started/control/authentication/sso-providers/overview Configure Single Sign-On with your Identity Provider Forest supports SAML 2.0 SSO. The configuration is done in two steps: 1. **Declare Forest in your Identity Provider** using the values below 2. **Configure Forest** with your IdP metadata You must be an Organization Owner to configure it. ## Forest SAML settings Use these values when configuring Forest as a Service Provider in your IdP: | Setting | Value | | ---------------------- | ----------------------------------------------- | | Callback URL (ACS URL) | `https://api.forestadmin.com/api/saml/callback` | | Sign on URL | `https://api.forestadmin.com/api/saml/callback` | | Logout URL | `https://app.forestadmin.com/login` | | Audience (EntityID) | Displayed in your Forest Organization settings | ## Configuration methods ### Option 1: XML metadata (recommended) Provide either a URL to your IdP's metadata XML endpoint, or upload the metadata XML file. This method supports automatic certificate rotation without service interruption. ### Option 2: Manual input Enter the following fields manually: * Login endpoint * Logout endpoint * Valid certificate ## Enabling SSO After configuring and testing your SSO setup, enable it for all users in your Organization settings. After enabling SSO, all users will be required to log in again. ## IdP-initiated login (optional) To allow users to be automatically redirected to Forest from your IdP dashboard, enable **IdP-initiated login** and set a default Relay State on your IdP: ```json theme={null} { "organizationName": "YourOrganizationName", "destinationUrl": "organization.projects" } ``` ## Troubleshooting * Double-check all endpoints and certificate expiration dates * Ensure the `NameID` configured on your IdP matches the **email address used on Forest accounts** * Ensure you selected **SAML 2.0** on your IdP ## Provider guides # Roles & Permissions Source: https://docs.forest.app/get-started/control/roles-permissions Create and manage custom roles with granular permission controls in Forest ## Overview Forest enables admins to create and manage custom roles with granular permission controls. The roles system allows organizations to define what actions users can perform within the platform. Only users with Admin permission level can create and manage roles. Roles are configured in the Roles tab within project settings, where permissions apply to all users assigned to that role. ## Permission levels Forest defines five user permission levels with increasing administrative capabilities: | Capability | User | Manager | Editor | Developer | Admin | | -------------------------------------- | ---- | ------- | ------ | --------- | ----- | | **Data Management** | ✓ | ✓ | ✓ | ✓ | ✓ | | Browse collections | ✓ | ✓ | ✓ | ✓ | ✓ | | View, create, update, delete records\* | ✓ | ✓ | ✓ | ✓ | ✓ | | Execute actions\* | ✓ | ✓ | ✓ | ✓ | ✓ | | **Inbox Management** | | ✓ | ✓ | ✓ | ✓ | | Manage inbox and notifications | | ✓ | ✓ | ✓ | ✓ | | **UI Customization** | | | ✓ | ✓ | ✓ | | Customize layouts and views | | | ✓ | ✓ | ✓ | | Configure collection displays | | | ✓ | ✓ | ✓ | | Create and edit Smart Views | | | ✓ | ✓ | ✓ | | Create workspaces | | | ✓ | ✓ | ✓ | | Create dashboards | | | ✓ | ✓ | ✓ | | Create workflows | | | ✓ | ✓ | ✓ | | **Environment Management** | | | | ✓ | ✓ | | Manage environments | | | | ✓ | ✓ | | Configure environment settings | | | | ✓ | ✓ | | Deploy between environments | | | | ✓ | ✓ | | **Team & Role Management** | | | | | ✓ | | Manage teams and users | | | | | ✓ | | Create and manage roles | | | | | ✓ | | Configure project settings | | | | | ✓ | | Access all administrative features | | | | | ✓ | \*Based on collection and action permissions configured for the role ## Roles Roles are configured in **Project Settings → Roles**. Each role defines granular permissions for collections and actions. ### Collection permissions Control what users can do with data in each collection: | Permission | Description | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------- | | **Read (List)** | Access to table view data - allows users to see the list of records in a collection | | **Read (Details)** | Access to details and summary view data for individual records - allows users to open and view a specific record's complete information | | **Create** | Record creation capability, including the ability to duplicate existing records | | **Update** | Modify existing records | | **Delete** | Remove records permanently | | **Export** | Export data from the collection in various formats (CSV, JSON, etc.) | ### Action permissions Control user ability to trigger and approve actions: | Permission | Description | | -------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Trigger** | Allow users assigned to this role to trigger this action - the basic execution permission | | **Require Approval** | Actions won't execute without manual approval - creates an approval workflow where actions are queued and must be reviewed before execution | | **Approve** | Permits role members to approve trigger requests submitted by other users - can review and authorize pending action requests | | **Self Approve** | Allows users to approve their own action requests - bypasses the need for another user to review the action, useful for trusted users who need faster execution | ### Approval workflows When "Require Approval" is enabled for an action: 1. User triggers the action 2. Action enters pending state in the approval queue 3. Users with "Approve" permission review the request 4. Action executes once approved (or is cancelled if rejected) Approval lifecycle: a triggered action becomes Pending, then either Approved (and Executed) or Cancelled This workflow ensures sensitive operations (like refunds, data exports, or account deletions) go through proper review before execution. [Learn more about action approval workflows →](/product/collaborate/approval-workflows) ## Conditional permissions Restrict permissions based on data conditions using filters. For example, operators might trigger refunds under \$1,000 without approval, while higher amounts require authorization. Conditional permissions allow you to: * Set data-based filters on any permission * Create dynamic permission rules based on field values * Combine multiple conditions with AND/OR logic * Apply different permission levels based on record data **Example:** Allow the "Support" role to trigger refunds only when `amount < 1000`, otherwise require approval. ## Default permissions Configure default permissions that are automatically applied when new collections or actions are created. This ensures consistent permission settings across your project without having to manually configure each new collection or action. Default permissions are configured in **Project Settings → Roles** and apply to: * New collections added to your datasources * New actions created in the back-end * New fields added to existing collections This saves time and ensures security by establishing baseline permissions that new elements inherit automatically. # Scopes Source: https://docs.forest.app/get-started/control/scopes-and-visibility Control which records users can see with dynamic data filtering ## What is a Scope? A scope functions as "a filter which applies to a collection and all its segments." Scopes enable dynamic data filtering based on the current user, allowing organizations to control which data different users can access throughout the application. **Scope Limitations**: Scopes apply to the entire application except for global actions, API & SQL charts, and Collaboration & Activities sections. ## Setup process To configure scopes, navigate to a collection's settings page using the Layout editor mode, then access the Scopes tab. Users can establish a filter and save it to restrict visible data. For example, filtering for customers whose email addresses end with "@forestadmin.com" would limit collection display to only those matching records. ## Dynamic scopes with user variables Scopes support dynamic filters based on user attributes. Available dynamic variables include: * `$currentUser.id`, user identifier * `$currentUser.firstName` / `lastName` / `fullName`, user name data * `$currentUser.email`, user email address * `$currentUser.team.id` / `team.name`, team information * `$currentUser.tags.your-tag`, custom user tag values ### Practical example An organization with regional operations teams (France team, Germany team) can filter by team name. When Marc from the France team logs in, he sees only French customer data. Louis from the Germany team simultaneously sees only German customer data. ## User Tags When user data doesn't directly match database records, user tags provide a solution. Administrators assign tags to individual users through their details pages, then reference those tags in scope filters using `$currentUser.tags.your-tag` syntax. This creates flexible mappings between users and accessible data. # Two-Factor Authentication Enforcement Source: https://docs.forest.app/get-started/control/security/2fa-enforcement Enforce 2FA for specific environments ## Overview Two-Factor Authentication (2FA) enforcement allows organizations to enforce a second authentication factor while login to specific environments. ## How 2FA Enforcement Works While 2FA is available platform-wide in Forest, the Security tab allows you to make 2FA mandatory before users can access specific environments. ## Third-Party Login Providers For users who log in through third-party providers, ensure that 2FA is enabled on that provider to maintain security parity. When using third-party login providers (such as Google, Microsoft, etc.), the 2FA security must be configured at the provider level to ensure consistent protection. ## Configuration To enforce 2FA for specific environments: Go to **Project Settings** > **Security Tab** Select the environments where 2FA should be mandatory Save your configuration to enforce 2FA requirements Once configured, users will be required to complete two-factor authentication before accessing the specified environments. # Security & Privacy Architecture Source: https://docs.forest.app/get-started/control/security/architecture Understand how Forest protects your data with privacy-first architecture and robust security measures Forest is built with security and privacy at its core. Your data never transits through Forest servers, and you maintain complete control over your infrastructure and access policies. ## Data privacy ### Private by design Forest implements a **privacy-first architecture** where your data flows directly between your Back-end and user browsers, never passing through Forest servers. **How it works:** When users access the Forest UI, their browser establishes two separate connections: 1. **Forest servers:** Retrieves layout configuration, UI settings, and metadata 2. **Your Back-end:** Retrieves actual data from your database Forest security and privacy architecture diagram **What Forest sees:** * UI layouts and configurations * User authentication metadata (email, role, permissions) * API request logs (endpoints called, timestamps) **What Forest never sees:** * Your actual data (customer records, transactions, etc.) * Database credentials * Your `FOREST_AUTH_SECRET` This architecture ensures your data remains within your infrastructure at all times. ### No third-party tracking Diagram showing no third-party data sharing Forest guarantees data privacy across all plan levels: * **No data sharing:** Your data is never sold or shared with third parties * **No third-party analytics on data:** Forest doesn't track or analyze your business data * **Optional tracking control:** Organizations can disable third-party vendors that might track activity metadata from browsers ## Security measures ### Token-based authentication Forest uses a **dual-token authentication system** to secure both UI access and Back-end communication. Forest authentication credentials flow #### FOREST\_ENV\_SECRET Authenticates requests between your Back-end and Forest servers. **Purpose:** * Links your Back-end to your Forest project * Authenticates layout and configuration requests * Required for all architectures (Cloud, Self-Hosted, On-Premise) **Security notes:** * Generated by Forest * Unique per environment (development, staging, production) * Should be stored as an environment variable ```bash .env theme={null} FOREST_ENV_SECRET=1234567890abcdef1234567890abcdef1234567890abcdef ``` Never commit `FOREST_ENV_SECRET` to version control. Always use environment variables or secret management tools. #### FOREST\_AUTH\_SECRET Authenticates requests between user browsers and your Back-end (Self-Hosted and On-Premise only). **Purpose:** * Signs JWT tokens for user authentication * Validates requests to your Back-end * **Your choice** - Forest never knows this secret **Security notes:** * Generated by you (not Forest) * Should be at least 32 characters long * Unique per environment * Used only in Self-Hosted and On-Premise architectures ```bash Generate a secure secret theme={null} # Using Node.js node -e "console.log(require('crypto').randomBytes(32).toString('hex'))" # Using OpenSSL openssl rand -hex 32 ``` ```bash .env theme={null} FOREST_AUTH_SECRET=your-secure-random-string-at-least-32-characters-long ``` **Cloud architecture:** `FOREST_AUTH_SECRET` is not needed because authentication is handled by Forest servers. Your data still flows directly from your Back-end to browsers without passing through Forest. ### JWT token structure Both tokens are JSON Web Tokens (JWT) containing user context: JWT token flow between Forest and your agent **Token payload includes:** * User ID * Email * Full name * Role * Team * Tags * Permissions **Use cases for token data:** * Custom authorization logic in your Back-end * Audit logging * Dynamic filtering based on user context * Integration with your internal systems ```javascript Node.js - Access user context theme={null} agent.customizeCollection('orders', collection => { collection.addHook('Before', 'List', async (context) => { const { email, role } = context.caller; // Custom logic based on user context if (role !== 'admin') { context.filter = { user_email: email }; } }); }); ``` ```ruby Ruby - Access user context theme={null} collection.add_hook(:Before, :List) do |context| email = context.caller.email role = context.caller.role # Custom logic based on user context unless role == 'admin' context.filter = { user_email: email } end end ``` ### Infrastructure flexibility You maintain **complete control** over your Back-end deployment: Forest agent deployment behind DMZ and VPN **Deployment options:** * **DMZ (Demilitarized Zone):** Deploy Back-end in isolated network segment * **VPN:** Require VPN connection to access Back-end * **Private Cloud:** Deploy within your private cloud infrastructure * **On-Premise:** Keep everything within your data center **Network security:** * Configure firewall rules * Set up network segmentation * Implement reverse proxies * Use TLS/SSL for all connections **Best practice:** Deploy your Back-end behind a VPN or firewall to add an additional layer of security. Even if an attacker obtains valid credentials, they would still need network access to reach your Agent. ### HTTPS/TLS encryption All communication is encrypted: * **Browser ↔ Forest:** HTTPS with TLS 1.2+ * **Browser ↔ Your Back-end:** HTTPS (you configure) * **Back-end ↔ Forest:** HTTPS with TLS 1.2+ Always deploy your Back-end with HTTPS enabled in production. Never use HTTP for sensitive data. # Auto Logout Source: https://docs.forest.app/get-started/control/security/auto-logout Force user session logout after inactivity ## Overview Auto Logout allows organizations to force your users session logout after a few minutes of inactivity. ## Available Timeout Options You can configure the following inactivity timeout periods: * 1 min * 2 min * 3 min * 10 min * 30 min * 1 hour (default) * 2 hours * 3 hours * 4 hours * 5 hours * 10 hours * 24 hours ## Configuration To configure Auto Logout settings: Go to **Project Settings** > **Security Tab** Select your desired timeout period from the available options Save your configuration to apply the auto logout policy # IP Whitelisting Source: https://docs.forest.app/get-started/control/security/ip-whitelisting Control which IP addresses can access your back-office ## Overview IP Whitelisting enables admins to control which IP addresses can access your back-office. This feature restricts back-office access to approved IP addresses only. ## How it works When IP Whitelisting is enabled, Forest restricts access to your back-office to only those IP addresses that have been explicitly whitelisted. Users attempting to access from non-whitelisted IP addresses will be blocked. IP whitelisting architecture ## Configuration To configure IP Whitelisting: Go to **Project Settings** > **Security Tab** > **IP Whitelisting** Turn on the IP Whitelisting feature Add the IP addresses that should have access to your back-office Save your configuration to enforce IP restrictions # Teams Source: https://docs.forest.app/get-started/control/teams Give each group its own layout: a tailored interface, dashboards, and data visibility per team ## Overview Teams allow you to create different layouts for different groups within your organization. Each team can have its own customized interface, optimized for their specific workflows and priorities. **Example use cases:** * **Customer Support** might prioritize displaying transaction IDs in the first column and default to the Data tab * **Sales team** would prefer showing company names first and default to the Dashboard tab * **Operations** could have a layout focused on order fulfillment metrics and status tracking Teams enable you to mirror your internal organizational structure within Forest, ensuring each department sees the most relevant information for their daily work. When combined with scopes, teams enable row-level security where users only see data relevant to their team context. Teams are managed in **Project Settings → Teams**. ## Create a team To establish a new team, navigate to the Teams tab within your project settings and select "+ New team". When creating a team, you have the option to "optionally copy another team's layout". ## Team layout management A team's layout encompasses all visible UI aspects, customizable per team: * **Workflows** - Custom workflows specific to the team * **Inbox** - Team-specific inbox configuration * **Workspaces** - Organized collections and navigation * **Collection settings and widgets** - Including Smart Views, columns, and table configurations * **Actions visibility** - Which actions are visible to the team * **Dashboards** - Custom dashboards with team-relevant metrics * **Record settings** - Summary views, analytics per record, related data tabs To apply an existing team's configuration to another team, access the team's settings page and use the "Copy" feature. This action is irreversible, so review carefully before proceeding. ## Delete a team To remove a team, go to its settings page, scroll beneath the user list, and click "Delete this team". You'll be asked to retype the team name as a confirmation step. This is a permanent action that cannot be undone. ## Advanced use cases Teams unlock advanced use cases for different organizational needs: * **Customer portals** - Create dedicated teams for external customers with limited, customized access to their own data * **Multi-tenant applications** - Isolate data and interfaces for different clients or business units * **Partner access** - Provide partners with tailored views into relevant data without exposing everything # Browse & customize your data Source: https://docs.forest.app/get-started/customize-your-back-office Explore your records, then shape collections and fields into a back-office your team can actually use Out of the box, every table in your database is surfaced as a collection with full browse, search, and edit. This step is about exploring what you have, then making it useful for the people who'll work in it every day. ## Browse and search your data