Ask HN: What tech to run and monitor CRON jobs (Python, Node.js, C#)?

LikeAnElephant 2021-08-17 15:57:44 +0000 UTC [ - ]

Regarding monitoring: I've come to rely on healthchecks.io to ensure my cron jobs when like I expect.

jorislacance 2021-08-18 07:20:53 +0000 UTC [ - ]

Monitoring solved, how do you deal with the CICD/delivery of your CRON code then?

ezekg 2021-08-17 19:25:59 +0000 UTC [ - ]

I run cron jobs using sidekiq-cron and monitor them using https://cronitor.io.

jorislacance 2021-08-18 07:15:20 +0000 UTC [ - ]

I found it quickly, but language tied (ruby in this case) :/

threecheese 2021-08-17 18:52:05 +0000 UTC [ - ]

Many teams use Rundeck for scheduled jobs and runbook automation. It gives you a nice UI and a measure of governance around the process. https://docs.rundeck.com/docs/learning/

jorislacance 2021-08-18 07:17:11 +0000 UTC [ - ]

I didn't know this one, I'll take a look at it, thanks!

zbentley 2021-08-17 17:01:01 +0000 UTC [ - ]

https://deadmanssnitch.com/

chudi 2021-08-17 22:24:06 +0000 UTC [ - ]

Rundeck is a pretty good tool to schedule, save logs and the like.

At my work we have a rundeck server connect via ssh to differents vm and execute the script there, you can configure alarms, parallel execution, plenty of steps it's pretty good you should try it out.

jorislacance 2021-08-18 07:17:00 +0000 UTC [ - ]

I didn't know this one, I'll take a look at it, thanks!

aprdm 2021-08-18 07:21:33 +0000 UTC [ - ]

Jenkins can work quite well for this. It keeps logs, success/failure, has hooks for multiple notifications and etc.

jorislacance 2021-08-18 07:27:42 +0000 UTC [ - ]

that's sadly what we do nowadays, but this is quite an hack somehow; there are hard limits in the monitoring features, not a lot of overview analytics viz too.

Our core issue is that it is a too much tech software, we would like to have PM/PO looking at the health status of things. We could build a dashboard about jobs status elsewhere for sure. But we are hoping to find a better tech to deliver, run and monitor jobs, all in one.

aprdm 2021-08-19 04:30:28 +0000 UTC [ - ]

You can template/customize the job view quite a lot in jenkins with HTML. We found it an OK compromise.

It has prometheus plugins for monitoring

jorislacance 2021-08-19 08:14:59 +0000 UTC [ - ]

Woh! golden feedback, we should look at doing this too. Appreciate your follow up thanks you!

rozenmd 2021-08-17 22:34:03 +0000 UTC [ - ]

There are SaaSes for this, like https://cronhub.io

jorislacance 2021-08-18 07:28:52 +0000 UTC [ - ]

We are considering. but it makes us migrate all 'stupid' jobs into API that can be triggered from cronhub.io.

vyrotek 2021-08-17 16:13:19 +0000 UTC [ - ]

How long of a job do you need to run?

We use Azure Timer Functions and/or Logic Apps to run and monitor our jobs. All of those languages are supported.

jorislacance 2021-08-18 07:12:09 +0000 UTC [ - ]

From 5 minutes to 24h mainly.

Some jobs have crazy deps, like puppeteer or selenium, I wonder if you can setup complex environment dependencies with FaaS (docker base image maybe?)

recov 2021-08-17 17:24:42 +0000 UTC [ - ]

A little overkill, but airflow. Free, has history, language agnostic, retries, can handle a large number of time based tasks.

jorislacance 2021-08-18 07:18:34 +0000 UTC [ - ]

Very big data alternative!

rahimnathwani 2021-08-17 21:58:50 +0000 UTC [ - ]

crontab to run them

dogwrap (command-line tool for datadog) to monitor them

jorislacance 2021-08-18 07:23:41 +0000 UTC [ - ]

interested about dogwrap, seems very generic for a various cases.

How do you deal with the CICD/delivery of your CRON code then using crontab?