Skip to main content

Listeners

Saffron supports listeners for various event. Listeners can be used for logging or creating analytics.

Database

database.set.okay

Arguments: source, articles

It is called when the articles are successfully uploaded to the database.

database.set.error

Arguments: source, error

It is called when an error is occurred when uploading the articles.

database.get.error

Arguments: source, error

It is called when an error is occurred when requesting articles from the database.

Scheduler

scheduler.path.error

Arguments: error

It is called if there is an error when reading the source file directory. Common errors are the directory does not exist or there are insufficient permissions.

scheduler.sources.new

Arguments: sourceNames

It is called when the scheduler loads the source files.

info

Note that this listener will be called after includeOnly and exclude configuration options are applied.

scheduler.sources.error

Arguments: sourceFile, error

It is called when the scheduler failed to load a source file.

scheduler.job.new

Arguments: jobId

It is called when the scheduler pushes a new source scrapping job at the stack.

scheduler.job.finished

Arguments: jobId

It is called when the scheduler finds a finished source scrapping job at the stack.

scheduler.job.failed

Arguments: jobId

It is called when the scheduler finds a failed source scrapping job at the stack.

scheduler.job.reincarnate

Arguments: job

It is called after the scheduler finds a failed source scrapping job and pushes it back to the stack.

scheduler.job.push

Arguments: job

It is called after the scheduler finds a failed source scrapping job and pushes it back to the stack.

scheduler.job.worker.replace

Arguments: oldWorkerId, job

It is called when scheduler finds a source scrapping job that has to been sent to the workers.

Grid

grid.node.connected

Arguments: socket

It is called when a new node is connected to the Grid.

grid.node.disconnected

Arguments: socket

It is called when a node disconnects from the Grid.

grid.worker.announced

Arguments: workerId

It is called when a worker is available to take source scrapping jobs.

grid.worker.destroyed

Arguments: workerId

It is called when a worker stops being available to take source scrapping jobs.

Workers

workers.job.finished

Arguments: job

It is called when a worker has finished a source scrapping job.

info

Note there may be a max of checksInterval ms timeframe with the scheduler.job.finished event.

workers.job.failed

Arguments: job

It is called when a worker has failed a source scrapping job.

info

Note there may be a max of checksInterval ms timeframe with the scheduler.job.failed event.

workers.articles.found

Arguments: articles, tableName

It is called after a source scraping job has finished and before middleware has been called. It gives as parameters the found articles from the website and the sourceName.

workers.articles.new

Arguments: articles, tableName

It is called after saffron has checked with previous articles stored in the database. It gives as parameters the new articles that will be uploaded to the database.

workers.parsers.error

Arguments: error

It is called if a source scraping job has failed.

Middleware

middleware.before

Arguments: articles

It is called before middlewares are applied and after workers.articles.found. The difference between the two is that middleware.before will not be called in case of zero articles.

middleware.after

Arguments: articles

It is called after middlewares are applied and before workers.articles.new.

Catch All

To catch all the events:

saffron.on("*", (eventName, ...args) => {
// ...
console.log(args)
});