Introduction
Saffron | News & announcements aggregation framework
Simple Abstract Framework For the Retrieval Of News
Status: Beta
The project is maintained by po/iw (ฯฮฟฮนฯ), a Greek university team that supports Free/Libre and Open Source Software and hardware. Saffron was built to help teams collect and compile huge lists of aggregated news and announcement content intuitively and efficiently.
What is this?
Saffron is an abstraction engine that helps you collect news and announcements from websites in a uniform way. All you have to do is write a file for each website you want to be scraped and run it in your NodeJS app. Saffron manages the scraping interval, data parsing and - if you want to - the data offloading to your database of choice.
Architecture
Saffron's architecture is based on a main
node that issues scraping instructions and several worker
nodes
that do the scraping & upload the data to the database.
The communication between the nodes is happening through the Grid
. By using a socket.io
implementation and in app encryption system, the communication is end to end safe.
Versions
2.1.0
- New features
- Added event listeners
middleware.before
&middleware.after
- Added log levels
info
&errors
- Added event listeners
2.0.0
- Changes
- Reformatted configuration fields.
- Reformatted source fields.
- New features
- Added more event listeners
- Logs can be switched off.
1.x.x
Version 1.x.x
was created for test purposes, and it is quite unstable. It is recommended that you're using any version above 2.0.0
.