Python script to generate lists of code scanning queries in CSV format#4177
Conversation
Creates a PowerShell script that can be used to report on the set of queries inside of a particular QL Suite.
Create query-list.yml
Add some debugging statements to workflow
|
Thanks for the thorough review, @adityasharad! I've addressed your comments and I think this is now ready for another look. Note that I've also set up the Action to no longer run on all pushes, so any changes to this branch won't immediately trigger a run of the script! |
|
👋 just driving by to say that we might to publish these query lists the along side the CodeQL query help eventually--we're currently looking at creating a new process for publishing the query help so that we can plan to move off Confluence sooner rather than later (see https://github.com/github/semmle-docs/issues/93#issuecomment-687797541). |
|
@hmakholm: I've made some changes following your suggestions. Could you and/or @adityasharad have a final look please? Thasks! |
|
Hold on, the Action is reporting: Just debugging |
adityasharad
left a comment
There was a problem hiding this comment.
Looks good! Couple of minor suggestions on the Actions trigger.
Co-authored-by: Aditya Sharad <6874315+adityasharad@users.noreply.github.com>
To confirm: this problem was fixed by fc8f01e. I've attached the most recently generated query list: code-scanning-query-list.zip |
|
Thanks @adityasharad, great suggestions. All implemented. Could you take another look please? |
adityasharad
left a comment
There was a problem hiding this comment.
Looks good. I suggest we squash and merge.
github#4177) * Create a PowerShell script that can be used to report on the set of queries inside of a particular QL Suite. * Translate PowerShell script into Python * support running this script from anywhere within the CodeQL git repo * print non-fatal error if metadata is not available * make sure warning about missing pack is printed to stderr * only run on pushes against main and rcs * detect repo by checking remote, rather than first SHA * specify full sha of dsaltares/fetch-gh-release-asset * trigger workflow on PR that modifies paths of interest Co-authored-by: Justin Hutchings <jhutchings1@users.noreply.github.com> Co-authored-by: Aditya Sharad <6874315+adityasharad@users.noreply.github.com>
tl;dr @jhutchings1 started working on a PowerShell script that would generate a CSV file containing metadata for all queries in code scanning query packs. This is a Python translation of that script. See #3664 for the PowerShell script.
Back in #3664, I pointed out some issues with that particular approach using PowerShell:
I therefore embarked on translating the script to Python back in mid-July. That all worked, but we never actually got to merge it. There were some issues with getting it to run on Actions, but that's all resolved now.
On the long term, we might want to move functionality like this into the CodeQL CLI (for one thing, the current approach is very slow), but the code scanning team are under some pressure to make this data available now (not in the least so it can be used for documentation purposes). I therefore suggest that we consider this script (and Actions workflow) an MVP for us to learn from: if this functionality turns out to be useful enough to have it in the CodeQL CLI, then we can do that later.
For ease of review: this is the (zipped) CSV data generated by the script, and here's the same data imported into Google Sheets.