Hugo Hacker News

Automating a Software Company with GitHub Actions

danpalmer 2021-08-19 14:50:25 +0000 UTC [ - ]

One of the things I like about Actions is how much it's focused on automation rather than CI. The pain points I've had with Circle/GitLab/Travis have often boiled down to the fact that they are often very specifically about _testing software_, not _automating processes_, and not even _deploying software_.

On that last one, there's a potential bug in the deployment pipeline here – deploys could run simultaneously or some bad luck on runner speed could even see an older version of the code go out after a newer version. Combined with the automated database migrations this could be quite a big problem!

Actions thankfully solved this recently with the `concurrency` key that lets you form a serial queue by a given key such as the branch name.

sytse 2021-08-19 15:50:19 +0000 UTC [ - ]

In GitLab you can limit concurrency by using a resource group https://about.gitlab.com/blog/2020/01/21/introducing-resourc...

Does that address your need?

Edit with more context: We use CI for deploying to GitLab.com and use resource_group to prevent multiple jobs from running concurrently. What we lack is the ability to prevent multiple pipelines from running concurrently (resource_group is at the job level). It looks like concurrency for actions https://docs.github.com/en/actions/reference/workflow-syntax... can work on groups which is a bit nicer. There is some discussion about making this better for GitLab in https://gitlab.com/gitlab-org/gitlab/-/issues/217522

Philip-J-Fry 2021-08-19 17:00:33 +0000 UTC [ - ]

I've never felt like GitLab CI is more about testing software than anything else...

It's super flexible and you can do literally anything.

verdverm 2021-08-19 15:13:39 +0000 UTC [ - ]

I'm not sure the concurrency key can solve the issue of old code after new code, in the general case.

What happens if there are conflicting migrations on two "parallel" branches?

What happens in you bad luck situation when commits are pushed in rapid succession on the same branch?

duped 2021-08-19 15:53:54 +0000 UTC [ - ]

I just want to be able to write all my workflow code as typescript (including the config - no YAML, for the love of god, no more YAML!) and run it locally with a debugger attached.

It's cost me hundreds to thousands of dollars to implement nontrivial workflows because of how the YAML is parsed (for example, empty strings when using a secret that has been renamed or removed) and the lack of introspection or debuggability when something goes wrong.

It's gotten to the point where new any new workflows I write are thin wrappers around a single script and I don't import any actions besides actions/checkout (even that has been bug prone, historically).

All that said, it's not like other platforms are better. But they certainly are cheaper and don't have dumb breakages when you need cross platform builds (has upload-artifact been fixed for executables on MacOS yet?)

wbond 2021-08-19 16:32:29 +0000 UTC [ - ]

Having dealt with a fair bit of CI on a decent number of services (AppVeyor, Travis, CircleCI and GitHub Actions), I’ve come to the exact same conclusion.

CI is a script, and the YAML configs for those various services configure the machine type, OS and toolchain. Everything else is contained within the script. Sometimes even toolchain setup is handled by the script.

Not following this model has wasted so much time when migrating services or trying to tweak what CI does.

With a script you can run it locally to ensure it performs the steps desired, leaving the CI “setup” to minimal environment/toolchain debugging.

i_s 2021-08-19 16:07:57 +0000 UTC [ - ]

Yea, making it so statements are composing via YAML was a very poor choice. One case I had was needing to change my actions so that steps D, E, and F could be retried together. What is the github YAML specification for decrementing a 'tries_remaining' variable, then going back to a previous step? I couldn't find it, so ended up just having to rewrite the action so that most of the steps are now in one bash script.

Not being able to execute it locally has also wasted a lot of time, and people needing to make 50+ changes to the master branch until they get it right.

MattGaiser 2021-08-19 16:46:44 +0000 UTC [ - ]

Yep. I am currently needing to try and set up code coverage reporting in GitHub (can't use Codecov as the company does not want to pay for anything) and it would be so much more trivial if we could just write regular code rather than futzing around with Bash and yaml.

minxomat 2021-08-19 17:01:25 +0000 UTC [ - ]

You can write custom actions in JavaScript, using the toolkit: https://docs.github.com/en/actions/creating-actions/creating...

However, the workflow referencing the custom action would still be the Actions YAML syntax.

(disclaimer: I work at GH, but not on Actions)

mrkurt 2021-08-19 16:40:46 +0000 UTC [ - ]

Do you have a picture of what typescript code for workflow would look like? Are you thinking of something that calls APIs directly or runs and spits out some kind of JSON instructions in place of yaml?

mirekrusin 2021-08-19 16:48:26 +0000 UTC [ - ]

Did anybody experiment with cuelang as source for GitHub actions yaml?

BugsJustFindMe 2021-08-19 15:25:16 +0000 UTC [ - ]

I love GH actions, but they're still a bit too sharp-edged for my taste.

Like...the last time I checked, workflows had no runtime macro for limiting execution to the default branch except explicitly by a specific name, and the closest you could get to generically checking "whatever the default branch is called right now" was either a template workflow that would set some static text for the name at creation that breaks if the default branch name is subsequently changed or a song and dance querying the API and setting an environment variable inside one of the workflow steps and then gating all subsequent steps on the result. This was a long time after they introduced editable default branch names and seems like such an obvious oversight.

Then there are weird quirks like the subshell file system permissions block that requires using sudo if you want to move files around within your repo clone from inside an invoked shell script.

mtalantikite 2021-08-19 15:50:36 +0000 UTC [ - ]

One relatively simple thing I'd love is the ability to re-run a job where it fails, which is something other CI systems have. On a current project we have some flaky Cypress tests and it sucks to have to re-run the entire workflow from the start when one of the last steps of the job fails. I'm definitely not the only one wanting this [1].

They also offer triggering workflows with 'workflow_run' based on other workflows, but that only happens on the default main branch. We auto build testing environments on each PR and I'd love to be able to have better workflow management based on branches.

[1] https://github.com/actions/runner/issues/432

c17r 2021-08-19 16:31:39 +0000 UTC [ - ]

I did a triple-take "...at PostHog we've been avid users of GitHub since its early ARPANET days."

bklyn11201 2021-08-19 15:55:07 +0000 UTC [ - ]

Can anyone point me to an example of a Github action resulting in standing up a fully working backend with a resolvable DNS entry for manual pull request testing?

reidjs 2021-08-19 15:14:37 +0000 UTC [ - ]

GitHub Actions are such a great tool, I use them to schedule tweets for me.

steeleduncan 2021-08-19 15:16:48 +0000 UTC [ - ]

Interesting, do you have a link to an example action for this?

reidjs 2021-08-19 15:20:02 +0000 UTC [ - ]

This should get you started :D let me know if you need help setting it up https://github.com/reidjs/markdown-tweet-scheduler

steeleduncan 2021-08-19 16:20:47 +0000 UTC [ - ]

Thanks for that, that looks great.

I've been looking for a way to post regularly without actually having to do it. tweetdeck is nice, but its still more hassle than filling a folder with markdowns.

2021-08-19 15:27:12 +0000 UTC [ - ]

ryanmarsh 2021-08-19 14:41:43 +0000 UTC [ - ]

LOL a software company is more, so much more, than CI. I thought I was going to read something novel about using GitHub actions for tracking sales leads or customer success or something.

spzb 2021-08-19 15:01:19 +0000 UTC [ - ]

This does seem to be a pretty bog-standard usage of a CI/CD tool. I must be missing something.

k__ 2021-08-19 15:16:16 +0000 UTC [ - ]

Seems like the article and also comments here imply just that.

GitHub Actions aren't focused on CI, so they are much more useful.