-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Swift: move TBD code to ql #9185
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
swift/ql/lib/codeql/swift/generated/typerepr/CompileTimeConstTypeRepr.qll
Fixed
Show fixed
Hide fixed
c12533d to
78bed81
Compare
|
There are quite a lot of code scanning complaints. What is the deal with those? |
These are because ql-for-ql doesn't know about our autogeneration scheme for Swift. I've written about it here. Since that's a private repo I'll copy/paste the relevant part: Since #9100 we added getAPrimaryQlClass for Swift the ql/primary-ql-class-consistency query currently (rightfully) reports a lot of inconsistencies. This is because we generate two classes for each ... thing ... in Swift. For instance, we have ApplyExpr and ApplyExprBase for a function call. The ApplyExprBase class is an autogenerated and private class that contains (among other things) the implementation of getAPrimaryQlClass, whereas ApplyExpr contains additional manually written predicates. This means that ApplyExprBase has a getAPrimaryQlClass predicate that returns "ApplyExpr" (instead of the expected "ApplyExprBase"). It should be straightforward to fix the ql-for-ql query to recognize this. The issue I wrote has two solutions, and Erik has suggested a third solution (and they should all be one-line fixes in ql-for-ql). |
78bed81 to
0452e2d
Compare
|
@sashabu or @jketema , can you have a look at the C++ part? If you start looking, can you immediately comment, so the other does not need to? And @MathiasVP, you might want to have a look at |
Thanks for the heads up. This LGTM 👍. |
0452e2d to
128ba22
Compare
Just to be clear: I started commenting on the QL part, which I am much more interested in. |
0d91158 to
83e1071
Compare
This allows to avoid bypassing label type correcness in the extractor, and allows to independently resolve TBD extractions, as with this approach TBD nodes do have the correctly typed trap label. The TBD status is now a predicate on the QL side. This requires: * a default visit using the correct type, which is achieved via macro metaprogramming in `VisitorBase.h`, following the way `swift::ASTVisitor` is programmed * a mapping from labels to corresponding binding trap entries. The functor is defined in `TrapTagTraits.h` and instantiated in generated `TrapEntries.h` * Binding trap entries for TBD unknown entities must not have any other field than the `id` (after all, we are supposed to not extract them yet). This is why all unextracted fields in `schema.yml` have been commented out, and will be uncommentend when visitors are added
83e1071 to
da00bf9
Compare
jketema
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally looks ok, but I think there are some details I'm missing. See questions below.
| trap.emit(UnknownAstNodesTrap{label, name}); | ||
| } | ||
| using Trap = BindingTrapOf<E>; | ||
| static_assert(sizeof(Trap) == sizeof(label), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ToBindingTrapFunctor seems pretty empty to me, so doesn't this always hold? I'm probably just missing details here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ToBindingTrapFunctor is implemented by TrapEntries.h in the generated headers. This particular assertion fires whenever a final class in schema.yml has a single property (i.e. non-optional, non-repeated). For example uncommenting line 995 in schema.yml (field introducer_int in class ConcreteVarDecl) and regenerating the dbscheme file will then result in
./swift/extractor/SwiftDispatcher.h:34:5: error: static_assert failed due to requirement 'sizeof(codeql::ConcreteVarDeclsTrap) == sizeof (label)' "Binding traps of unknown entities must only have the `id` field (the class should be empty in schema.yml)"
static_assert(sizeof(Trap) == sizeof(label),
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
which should point in the correct direction.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I still don't get this, because:
template<>
struct ToBindingTrapFunctor<TrapLabel<FileTag>> {
using type = FilesTrap;
};
pretty much looks like an empty class to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah, sorry, I did not get that that was your doubt.
It's the type nested alias that does the trick. From TrapTagTraits:
template <typename T>
using BindingTrapOf = typename detail::ToBindingTrapFunctor<TrapLabelOf<T>>::type;so for example BindingTrapOf<swift::VarDecl> will resolbe to typename detail::ToBindingTrapFunctor<TrapLabel<ConcreteVarDeclTag>>::type which in the generated file is defined as ConcreteVarDeclsTrap.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it. But then why do this at all and not just:
static_assert(sizeof(E) == sizeof(label),
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure but that's because there's actual work needed to implement
underlying_typeit's not just an alias like we have here.
I think that alias is there to hide away an implementation detail, which does the actual work in terms of template specialization.
Would you find it clearer if we wrote something like
template <typename Label>
using BindingTrapOf = typename detail::ToBindingTrap<Label>::type;and then in the code using it did:
using Trap = BindingTrapOf<decltype(label)>;or more explicitly
using Label = TrapLabelOf<E>;
using Trap = BindingTrapOf<Label>;
Label label = assignNewLabel(e);
...?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand less and less of what this is code is doing. Why is all this indirect mess needed at all? This is going to cause a lot of trouble for anyone who is going to maintain this in the future.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I want something that's just 2 lines of code instead of 12.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that should probably just be
TrapLabel<E>{label}.?
Hmm, no, labels cannot be emitted in place of trap entries. TrapLabelOf<E> which resolves to TrapLabel<TrapTagOf<E>> (for example, TrapLabel<ConcreteVarDeclTag>) denotes a typed trap label, so something that is a # label in the trap file. We need to emit an actual trap (like concrete_var_decls(#1e)) which in C++ is done printing a ConcreteVarDeclsTrap instance.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just for the record: @redsun82 and I discussed this face-to-face. For now I'm happy with this. I think it would be good to have another detailed look at how this is set up. My main issue is that a lot of the template programming being done here is needed to patch up the macro programming that the Swift compiler forces upon us, which feel rather brittle and difficult to maintain, due to this being split over quite a number of generated and manually implemented files. I wonder if this could be simplified significantly by moving towards more macro programming or moving towards more template programming.
This allows to avoid bypassing label type correcness in the extractor,
and allows to independently resolve TBD extractions, as with this
approach TBD nodes do have the correctly typed trap label. The TBD
status is now a predicate on the QL side.
The code taking care of making unknwon entities be printed as
TBD(Class)is in the hand written
Element.qllfile.This requires:
metaprogramming in
VisitorBase.h, following the wayswift::ASTVisitoris programmedfunctor is defined in
TrapTagTraits.hand instantiated in generatedTrapEntries.hfield than the
id(after all, we are supposed to not extract themyet). This is why all unextracted fields in
schema.ymlhave beencommented out, and will be uncommentend when visitors are added
closes https://github.com/github/codeql-c-team/issues/1024