Properties

Schemaregistry

Property Description Default

factcast.store.schema-registry-url

if a schema-registry-url is defined, FactCast goes into validating mode. The only protocols allowed here are "http", "https", "classpath" and "file". Note that http(s) and file always require two slashes after the colon, e.g. "https://someserver/…​" or "file:///root/folder/…​".

factcast.store.validation-enabled

Can be used for disabling Fact validation if a schema-registry-url is defined. Useful for mass data ingestion.

true

factcast.store.persistent-registry

if fetched Schema and Transformation Documents are persisted into Postgres

false

factcast.store.allow-unvalidated-publish

If validation is enabled, this controls if publishing facts, that are not validatable (due to missing meta-data or due to missing schema in the registry) are allowed to be published or should be rejected.

false

factcast.store.schema-store-refresh-cron

defines the cron schedule for refreshing the SchemaRegistry by querying for the latest remote changes

*/60 * * * * * (every minute)

factcast.store.allow-schema-replace

If a schema can be replaced by an updated version from the registry (not a good idea in production environments)

false


Transformation-Registry

Property Description Default

factcast.store.persistent-transformation-cache

if Transformed Fact payloads are persistently cached into Postgres.

false

factcast.store.in-mem-transformation-cache-capacity

when using the inmem impl of the transformation cache, this is the max number of entries cached. The minimum value here is 100.

100

factcast.store.delete-transformations-stale-for-days

when using the persistent impl of the transformation cache, this is the minimum number of days a transformation result is not read before it is considered stale. This should free some space in a regular cleanup job. Setting this to -1 disables cleanup of transformation results from the cache.

-1 (disabled)

factcast.store.transformation-cache-compact-cron

defines the cron schedule for compacting the transformation result cache.

0 0 0 * * * (at midnight)


Performance / Reliability

Property Description Default

factcast.store.fact-notification-blocking-wait-time-in-millis

Controls how long to block waiting for new notifications from the database (Postgres LISTEN/ NOTIFY mechanism). When this time exceeds the notifications is repeated. Minimum value is 5000.

15000 (15sec)

factcast.store.fact-notification-max-round-trip-latency-in-millis

When FactCast did not receive any notifications after fact-notification-blocking-wait-time-in-millis milliseconds it validates the health of the database connection. For this purpose it sends an internal notification to the database and waits for the given time to receive back an answer. If the time is exceeded the database connection is renewed. Minimum value is 50.

200

factcast.store.fact-notification-new-connection-wait-time-in-millis

how much time to wait between invalidating and acquiring a new connection. note: This parameter is only applied in the part of FactCast which deals with receiving and forwarding database notifications. Minimum value is 10.

100

factcast.store.page-size

How many Facts to fetch from the database in one go. Higher values mean more memory usage. This setting will be respected by all catchup strategies. Must be positive.

50

factcast.store.chunk-size

Size of a chunk, that is used to fetch events from the store during CHUNKED_WITH_HOLD catchup strategy. The page-size will still define how many rows will be loaded into memory.

10000

factcast.store.index-check-cron

Cron expression defining a routine check for index validity

0 0 3 * * * (3 am)

factcast.store.tail-indexing-enabled

enable/ disable tail indexing

false

factcast.store.tail-management-cron

cron schedule when tail rotation should be carried out

0 0 0 * * * (at midnight)

factcast.store.tail-generations-to-keep

the number of tail indexes to keep. The higher the number, the slower the inserts. Probably 2 or 3 is a good value unless you have a very high tail rebuild frequency and not permanently connected applications (like offline clients for instance). Must be a positive number, maximum is 128.

3

factcast.store.minimum-tail-age

minimum age of the youngest tail index, before a new one is created

7 days

factcast.store.tail-creation-timeout

Index creation can hang for a long time in case of many open transactions. To avoid this, you can specify a timeout.; We will subtract 5 seconds from the given duration before applying it to setTimeout.

1d

factcast.store.tail-indexing-fast-update-enabled

controls if, when creating tail indexes, fastUpdate feature will be used. Note that depending on the pending list, this may introduce pauses when inserting facts (flushing the list)

false

factcast.store.tail-indexing-pending-list-limit

maximum size of the pending list when using factUpdate. (See https://www.postgresql.org/docs/16/gin-tips.html)

4096

factcast.store.transformation-cache-page-size

Defines the max number of Facts being scheduled for transformation in one go. Must be positive and not exceed 32000.

100

factcast.store.transformation-cache-buffer-size

Defines the max number of transformed Facts being buffered in memory that will be flushed to the database in one go. Must be positive and not exceed 9999.

1000

factcast.store.size-of-thread-pool-for-subscriptions

This is the number of threads we create for handling new subscriptions requests. It’s implemented via a fixed thread pool. As soon as the subscription request finishes or enters phase 3 (follow) the thread is freed up again. In earlier versions we used the common FJP which limits the parallelism to the number of cores - 1. If you ever encounter too much database load or too high waiting time for subscriptions this can be an option.

100

factcast.store.size-of-thread-pool-for-buffered-transformations

This is the number of threads we create for handling buffered transformations. It’s implemented via work stealing thread pool. In early versions we used the common FJP which limits the parallelism to the number of cores - 1.

25

factcast.store.read-only-mode-enabled

Configures FactCast to work in read-only mode. You cannot publish any events in this mode and certain functionality like tail index generation or state token generation is disabled. You can still use a persistent schema store or transformation cache, however they will work in read-only mode. Additionally, liquibase is disabled.

false

factcast.store.enumeration-direct-mode-enabled

Despite of a Schema-Registry being defined or not, if set to true, enumeration of types or namespace will examine the data in the store directly, so that you only see data from already published facts.

false

factcast.store.auto-flush-delay

When catching up, if production of a full notification of facts takes longer than this value (in milliseconds), an additional flush is inserted into the pipelin in order to send the notification as is to the client. This is done in order to balance parallelization vs. network/compression efficiency.

10000

factcast.store.catchup-strategy

Available: CURSOR and CHUNKED_WITH_HOLD. Cursor does the catchup query in one go and keeps the cursor open until the facts are sent to the client. CHUNKED_WITH_HOLD first partitions serials into chunks and then fetched each chunk in a single query. Note that page-size still applies.

CURSOR

factcast.store.catchup-async-fetch

if true, fetching from the database happens async by splitting the page size in half and keep fetching one page ahead

false

factcast.store.log-suppression.enabled

In some situations logging can be overwhelming, when clients to rare, but expectedly long-running things like catching up from scratch. If enabled, this option limits the number of loglines created by that process.

false

factcast.store.log-suppression.min-log-level

Level to which the root log-level is raised during the operation: everything with this or above will be logged unaltered.

INFO

factcast.store.log-suppression.threshold

Number of logevents that will be regularly logged, before the suppresion kicks in.

1000

factcast.store.log-suppression.sample-rate

Even when suppressing, every N-th logevent can be logged nevertheless, to indicate that the process is still alive. 0 disables it.

1000

factcast.store.publish-batched.enabled

When enabled, the number of transactions can be reduced in burst publish situations. This increases throughput, but adds a bit of latency to the publish operation.

false

factcast.store.publish-batched.max-batch-size

Maximum number of publish operations combined into one. Trade-off between latency and efficiency.

500

factcast.store.offload.enabled

Authoritative toggle for catchup offloading. When true, a valid offload datasource configuration including factcast.store.offload.url is required; PHASE1 runs on that datasource and PHASE2 on the primary datasource. When false or omitted, offloading is disabled even if an offload URL or other datasource properties are configured.

false

factcast.store.offload.url

JDBC URL of the offload datasource. Required when factcast.store.offload.enabled=true. Configuring this URL alone does not enable offloading.

factcast.store.offload.username

same as any DataSource

factcast.store.offload.password

same as any DataSource

factcast.store.offload.driver-class-name

same as any DataSource

factcast.store.offload.type

same as any DataSource

factcast.store.offload.jndi-name

same as any DataSource

factcast.store.offload.generate-unique-name

same as any DataSource

factcast.store.offload.name

same as any DataSource


Snapshots

InMem-Snapshots

Property Description Default

factcast.snapshot.local.mem.delete-snapshot-stale-for-days

min number of days a snapshot is kept even though it is not read anymore. Must be a positive number.

90

InMemAndDisk-Snapshots

Property Description Default

factcast.snapshot.local.disk.path-to-snapshots

path to store the snapshots in the file system.

java.io.tmpdir

factcast.snapshot.local.disk.max-disk-space

max disk space to be used by the SnapshotDiskRepository. The oldest Snapshots will start to be removed after reaching 90% of the allocated space, never reaching 100%. Use 0 for unlimited disk space

0

RedisSnapshots

Property Description Default

factcast.snapshot.redis.delete-snapshot-stale-for-days

min number of days a snapshot is kept even though it is not read anymore. Must be a positive number.

90

JDBC-Snapshots

Property Description Default

factcast.snapshot.jdbc.delete-snapshot-stale-for-days

min number of days a snapshot is kept even though it is not read anymore. Must be a positive number.

90

factcast.snapshot.jdbc.snapshot-table-name

optional name of the table for the snapshots. When not provided the default will be used

factcast_snapshot

factcast.snapshot.jdbc.snapshot-access-table-name

optional name of the table for the snapshots access timestamp. When not provided the default will be used

factcast_snapshot_last_accessed

MongoDB-Snapshots

Property Description Default

factcast.snapshot.mongodb.delete-snapshot-stale-for-days

min number of days a snapshot is kept even though it is not read anymore. Must be a positive number.

90

Snapshot Serializers

Property Description Default

factcast.factus.snapshot.compress

Compress serialized snapshots before sending them to the snapshot-cache

true


gRPC

Properties you can use to configure gRPC:

gRPC Client

Property Description Default Example

grpc.client.factstore.credentials

Deprecated. Please use factcast.grpc.client.user and factcast.grpc.client.password instead

none

myUserName:mySecretPassword

spring.grpc.client.channel.factstore.target

the address(es) fo the factcast server

none

static://localhost:9090

spring.grpc.client.channel.factstore.ssl.enabled

Enables or disables SSL

false

true

spring.grpc.client.channel.factstore.keepalive.time

The default delay before sending keepAlives. Defaults to 60s. Please note that shorter intervals increase the network burden for the server.

300

spring.grpc.client.channel.factstore.keepalive.without-calls

Configures whether keepAlive will be performed when there are no outstanding RPCs on a connection.

false

true

spring.grpc.client.channel.factstore.keepalive.time=300
spring.grpc.client.channel.factstore.keepalive.without-calls=true

Further details can be found here : org.springframework.boot.grpc.client.autoconfigure.GrpcClientProperties.

FactCast client specific

Property Description Default Example

factcast.grpc.client.id

Server-side logging mentions this optional id if set in order to help with debugging. If this property is not set, it falls back to the value of spring.application.name

${spring.application.name}

myClient

factcast.grpc.client.user

User if factcast-security is enabled.

myUser

factcast.grpc.client.password

Password if factcast-security is enabled.

myPassword

factcast.grpc.client.enable-fast-forward

If the server supports it, enables fast forwarding. This is supposed to speedup frequent queries that cluster around the end of the global Fact-Stream and thus can use dedicated temporary rolling indexes.

true

false

factcast.grpc.client.max-inbound-message-size

The GRPC server will chunk messages to not exceed this. Note that one message might contain up to 1000 facts. This value disregards compression. The maximum for this value is 32MB, the minimum is 2 MB. Note that this also limits the maximum transaction bulk size.

3682304

16m

factcast.grpc.client.resilience.enabled

Enables resilience mode for subscriptions. If enabled, subscriptions that fail due to networking errors will be transparently resubscribed. (since 0.5.5)

true

false

factcast.grpc.client.resilience.window

Defines the window in which a maximum of retries is defined. (since 0.5.5)

PT30S (30 seconds)

PT2M (2 Minutes)

factcast.grpc.client.resilience.attempts

Defines the maximum number of attempts that will be done (within a time window defined by resilience.window) before failing and escalating the last exception to the application. (since 0.5.5)

10

25

factcast.grpc.client.resilience.interval

Defines the wait time between two attempts. (since 0.5.5)

PT0.1S (100 millis)

PT0.5S

factcast.grpc.client.ignore-duplicate-facts

Ignores and skips duplicate exceptions during publishing (does not include conditional publishing when using locks). This might be convenient in cases where you published to a factcast server and get a connection error back (you cannot possibly know if the publish on the server succeeded or not). If you have resilience enabled, the publish would be retried and might (if the first was successful) result in a DuplicateFactException. Setting this to true will make factcast just ignore the exception and go on. There might be a performance problem resulting from this: If you publish a batch of facts and a DuplicateFactException is received, factcast will fall back to publishing every single Fact from the batch one-by-one in order to make sure, that after your call, all Facts that are not duplicates will be published.

false

true

factcast.grpc.client.health-check-enabled

Enables a client side actuator health check that reports out of service as soon as the underlying connection reports a transient failure. You can achieve the same by excluding the GrpcClientHealthAutoConfiguration.class from your autoconfiguration.

true

false

grpc Server

Property Description Default Example

spring.grpc.server.keep-alive.permit-without-calls

Configures whether clients are allowed to send keep-alive HTTP/2 PINGs even if there are no outstanding RPCs on the connection

false

true

spring.grpc.server.keep-alive.permit-time

Specifies the most aggressive keep-alive time in seconds clients are permitted to configure. Defaults to 5min.

300

100

factcast.grpc.bandwidth.number-of-follow-requests-allowed-per-client-per-minute

after the given number of follow requests from the same client per minute, subscriptions are rejected with RESOURCE_EXHAUSTED

5

5

factcast.grpc.bandwidth.initial-number-of-follow-requests-allowed-per-client

ramp-up to compensate for client startup

50

50

factcast.grpc.bandwidth.number-of-catchup-requests-allowed-per-client-per-minute

after the given number of catchup requests from the same client per minute, subscriptions are rejected with RESOURCE_EXHAUSTED

6000

6000

factcast.grpc.bandwidth.initial-number-of-catchup-requests-allowed-per-client

ramp-up to compensate for client startup

36000

36000

factcast.grpc.bandwidth.disabled

completely disables checking if set to true

false

true

spring.grpc.server.keep-alive.permit-without-calls=true
spring.grpc.server.keep-alive.permit-time=100

Blacklist

Property Description Default Example

factcast.blacklist.type

Configures where the list of blacklisted facts is retrieved from. One of [POSTGRES, RESOURCE].

POSTGRES

---

factcast.blacklist.location

Only required if type=RESOURCE. Specifies the URL where the JSON file containing the blacklist is located.

classpath:blacklist.json

file:/some/path/blocked-facts.json


Testing

Property Description Default

factcast.store.integration-test-mode

when set to true, disables all non-essential memory-internal caches, timing might differ to production of course.

false


UI

Property Description Default

vaadin.production-mode

Should be set to true, otherwise vaadin tries to generate a dev bundle which is not necessary, and probably will fail.

false

factcast.ui.report.store.path

The path under which reports are stored if no external ReportStore is configured.

/tmp/factcast-ui/report

factcast.ui.report.store.s3

The name of the S3 Bucket in which the reports are stored by the S3ReportStore. This overrides factcast.ui.report.store.path

Further details can be found here : org.springframework.boot.grpc.server.autoconfigure.GrpcServerProperties.