Sitelet https://web.archive.org/web/20201219125623/https://github.com/bitcoin/bitcoin/issues/19815
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve scripted-diff check #19815

Open
amitiuttarwar opened this issue Aug 27, 2020 · 7 comments
Open

improve scripted-diff check #19815

amitiuttarwar opened this issue Aug 27, 2020 · 7 comments

Comments

@amitiuttarwar
Copy link
Member

@amitiuttarwar amitiuttarwar commented Aug 27, 2020

Context:

Scripted diffs are a way to automate some types of reformatting or refactoring changes. When a commit message matches a specific format (begins with scripted-diff:), CI will run commit-script-check.sh.

sed is platform dependent. One significant difference is when using the -i flag. On BSD (eg. macOS), you have to insert an empty string, but with GNU sed (used by Travis), this is compatible.

example of a working command for these two platforms-
macOS: sed -i '' 's/a oneshot/an addrfetch/g' src/chainparams.cpp
Travis: sed -i 's/a oneshot/an addrfetch/g' src/chainparams.cpp

Task:

Add logic to the commit-script-check.sh to detect if the scripted diff is using the BSD sed syntax and print a helpful error message.

Useful skills:

Bash

Want to work on this issue?

For guidance on contributing, please read CONTRIBUTING.md before opening your pull request.

@Z5483
Copy link

@Z5483 Z5483 commented Aug 27, 2020

You can also use append empty string directly after -i to make sed -i work on both GNU and BSD sed:

sed -i'' 's/a oneshot/an addrfetch/g' src/chainparams.cpp

I want to work on this, but I think it is good to recommend people to do the above for portability for now.

@amitiuttarwar
Copy link
Member Author

@amitiuttarwar amitiuttarwar commented Aug 28, 2020

@Z5483 hi! thanks for taking a look & opening a PR!

I tried this trick out locally (macOS), but got an error... actually, I'm getting different errors on each try 👀
(but doing -i '' is working fine)

my attempts, incase you're curious

example 1: sed -i'' 's/MempoolUnbroadcastTest/LaLaLaTest/g' test/functional/mempool_unbroadcast.py
error: sed: 1: "test/functional/mempool ...": undefined label 'est/functional/mempool_unbroadcast.py'

example 2: sed -i'' 's/CMainParams/SuperImportantParams/g' src/chainparams.cpp
error: sed: 1: "src/chainparams.cpp": unterminated substitute in regular expression

example 3: sed -i'' 's/MAX_FEELER_CONNECTIONS/MAX_FEELS/g' src/net.h
error sed: 1: "src/net.h": unterminated substitute pattern

but I see you've opened a patch to update the script, so I'll check that out :)

@robot-dreams
Copy link
Contributor

@robot-dreams robot-dreams commented Sep 16, 2020

@Z5483 Thanks for working on this! I've ran into some scripted-diff issues as well, and I'm very excited to see that you're improving it :)

@amitiuttarwar I'm a MacOS user as well, and I ran into the following issue; did you ever encounter this as well?

$ ./test/lint/commit-script-check.sh 85165d4~..85165d4
sed: 1: "/^-BEGIN VERIFY SCRIPT- ...": unexpected EOF (pending }'s)
Error: missing script for: 85165d4332b0f72d30e0c584b476249b542338e6
Failed

I fixed it locally by installing gsed and creating a symbolic link. But do you think it would make sense to add something like the following to help future MacOS users?

Screen Shot 2020-09-16 at 3 31 54 PM

After making this change, here's the new output:

$ ./test/lint/commit-script-check.sh 85165d4~..85165d4
sed: 1: "/^-BEGIN VERIFY SCRIPT- ...": unexpected EOF (pending }'s)
Error: Incompatible version of sed; MacOS users should install gsed
Failed
@bitcoin bitcoin deleted a comment from ArtemVinter Sep 19, 2020
@bitcoin bitcoin deleted a comment from m-s-32 Sep 19, 2020
@bitkarrot
Copy link

@bitkarrot bitkarrot commented Dec 6, 2020 •

I'm signing up for this task (if still open) because @bliotti wants me to work on it as "I'm way overdue for a bitcoin commit" as a Jimmy Song Alumni, using my public face here.

@bliotti
Copy link
Contributor

@bliotti bliotti commented Dec 6, 2020 •

@bitkarrot Awesome! Looks like a good one. @amitiuttarwar or @Z5483 where does this one stand? We would like to pick this one up if still available. Thanks!

@Z5483
Copy link

@Z5483 Z5483 commented Dec 6, 2020 •

Feel free to work on this. I scrapped my PR because I didn't put time into it so it's horrible. I'm busy with schools for the rest of the school year so I don't think I will pick this up again anytime soon. I hope you guys can produce something better than mine.

@wodry
Copy link
Contributor

@wodry wodry commented Dec 6, 2020

Catching sed syntax usage that leads to unexpected results is not a low hanging fruit I guess :)

Maybe you should pimp your own script like so:

kernel_name="$(uname -s)"

case "${kernel_name}" in
    Linux*)     sed_option='-i'
                ;;
    Darwin*)    sed_option='-i ""'
                ;;
    *)          echo "Unknown kernel name: '${kernel_name}'"
                exit 1
esac

sed $sed_option 's/a oneshot/an addrfetch/g' src/chainparams.cpp

With that, you can define all specialities needed for different environments. You could pimp it even more like checking the the version etc.

Maybe you could create a PR to add this as a template to https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#suggestions-and-examples

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.