Sitelet https://web.archive.org/web/20210805214512/https://github.com/ethereum/solidity/issues/11553
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CLI] Different source hashes when source is passed via standard input and via command line parameter #11553

Open
cameel opened this issue Jun 17, 2021 · 8 comments · May be fixed by #11641
Open

Comments

@cameel
Copy link
Member

@cameel cameel commented Jun 17, 2021

Description

The content of the file passed to standard input does not pass through unmodified and its content hash stored in metadata (note: I do not mean the metadata hash itself) is different than when the file name is given as an argument.

The problem is caused by our function for reading standard input, which replaces all newlines with \n and also adds a newline after the last line, even if originally there wasn't one.

In practice the consequences of this are probably minor because even if the source hashes end up being the same, metadata won't be - due to different file names (unless the file is called <stdin> which is unlikely). Still, I think that this behavior is unexpected by users and should be changed. It looks intentional though and might be a workaround for something so we should discuss it first.

Environment

  • Compiler version: 0.8.5
  • Operating system: Arch Linux

Steps to Reproduce

echo "contract C {}" > '<stdin>'
solc '<stdin>' --metadata

sources key from metadata output:

    "sources": {
        "<stdin>": {
            "keccak256": "0xbe86d3681a198587296ad6d4a834606197e1a8f8944922c501631b04e21eeba2",
            "urls": [
                "bzz-raw://7ef32f5b5f3b1ebe447f46969c01ce0e9190d0b90241cbc228255866326be8be",
                "dweb:/ipfs/QmZRepHCmeSSTfxZvKU5KS7pebBYT4khWhTnqh6GzBFU3M"
            ]
        }
    },
cat contract.sol | solc - --metadata

sources key from metadata output:

    "sources": {
        "<stdin>": {
            "keccak256": "0xec5627577fc5b8010229a1036827ef0739fb21bb05b7bf259947acdded145877",
            "urls": [
                "bzz-raw://2363616d36c4824b30b1698ffa5bc3443635a4d68eb4a93c7126b30b7716f0a1",
                "dweb:/ipfs/QmYENep2eioJEAMKZWozLW3fb2BHjNxCXSszbDzz2kfWJu"
            ]
        }
    },
@cameel cameel added this to New issues in Solidity via automation Jun 17, 2021
@chriseth
Copy link
Contributor

@chriseth chriseth commented Jun 21, 2021

Do we need the processing?

@chriseth
Copy link
Contributor

@chriseth chriseth commented Jun 21, 2021

Reading the source, I cannot imagine this being intentional.

@cameel
Copy link
Member Author

@cameel cameel commented Jun 21, 2021

I don't think we need it.

It looked like intentional to me because I can't see why else one would read a file line by line but still return the content. The only reason I can think of is to replace the line endings.

@chriseth
Copy link
Contributor

@chriseth chriseth commented Jun 22, 2021

I think this is because it is the most convenient way to read a file.

@cameel cameel moved this from New issues to Implementation Backlog in Solidity Jun 22, 2021
@TerranCivilian
Copy link
Contributor

@TerranCivilian TerranCivilian commented Jun 26, 2021

Can you assign this to me? I can try to tackle this tomorrow and report back.

@cameel
Copy link
Member Author

@cameel cameel commented Jun 27, 2021

@TerranCivilian You might want to start by looking at the function that reads data from standard input:

string solidity::util::readStandardInput()
{
string ret;
while (!cin.eof())
{
string tmp;
// NOTE: this will read until EOF or NL
getline(cin, tmp);
ret.append(tmp);
ret.append("\n");
}
return ret;
}

@TerranCivilian
Copy link
Contributor

@TerranCivilian TerranCivilian commented Jun 27, 2021 •

Thanks for the excellent writeup. After many hours of research and benchmarking, I believe I've produced an optimal solution (as seen here #11584).

@TerranCivilian
Copy link
Contributor

@TerranCivilian TerranCivilian commented Jul 8, 2021

Still working on this, mostly on weekends. Hope to wrap it up this weekend.

@cameel cameel changed the title Different source hashes when source is passed via standard input and via command line parameter [CLI] Different source hashes when source is passed via standard input and via command line parameter Aug 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment