Sitelet https://web.archive.org/web/20201125115708/https://github.com/plotly/falcon/issues/450
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Apache Drill connector without S3 #450

Open
julioasotodv opened this issue May 30, 2018 · 9 comments
Open

Feature request: Apache Drill connector without S3 #450

julioasotodv opened this issue May 30, 2018 · 9 comments

Comments

@julioasotodv
Copy link

@julioasotodv julioasotodv commented May 30, 2018 •

Hi,

First of all, thank you all for Falcon. It's awesome.

However, it looks like the Apache Drill connector only works with S3 buckets. Nonetheless, Drill itself is able to query from lots of other places (such as a local filesystem, HDFS, MongoDB...).

Adding generic support for Apache Drill (without the need for S3 credentials) would be great. It would expand Falcon capabilities a lot with just one connector.

Thank you!

@julioasotodv julioasotodv changed the title Apache Drill connector without S3 feature request: Apache Drill connector without S3 May 30, 2018
@julioasotodv julioasotodv changed the title feature request: Apache Drill connector without S3 Feature request: Apache Drill connector without S3 May 30, 2018
@n-riesco
Copy link
Collaborator

@n-riesco n-riesco commented Jun 5, 2018

@julioasotodv sorry for the late reply. At the moment, the connector for Apache Drill doesn't implement some the functionality other SQL connectors have in Falcon.

I've had a quick look at the documentation, and the missing functionality is readily available in Apache Drill, so this is an easy first-time contribution.

If no one has a go at implementing the missing functionality, I'll do it myself next month.

@n-riesco n-riesco self-assigned this Jun 5, 2018
@n-riesco n-riesco mentioned this issue Jun 5, 2018
3 of 10 tasks complete
@julioasotodv
Copy link
Author

@julioasotodv julioasotodv commented Jun 7, 2018

@n-riesco TBH I'm not that familiar with how Falcon works itself (I do not have much spare time neither...)

I will try to poke around a little bit

@n-riesco
Copy link
Collaborator

@n-riesco n-riesco commented Jun 7, 2018

@julioasotodv Great! If you need any help, please, post here.

The main change is to add the functions tables and schemas to https://github.com/plotly/falcon-sql-client/blob/master/backend/persistent/datastores/ApacheDrill.js and update SQL_DIALECTS_USING_EDITOR and PREVIEW_QUERY in https://github.com/plotly/falcon-sql-client/blob/master/app/constants/constants.js .

We've got some documentation on implementing a connector in https://github.com/plotly/falcon-sql-client/blob/master/NEW_CONNECTION.md .

And see the instructions to build Falcon in https://github.com/plotly/falcon-sql-client/blob/master/CONTRIBUTING.md ; basically:

$ git clone https://github.com/plotly/falcon-sql-client falcon-sql-client
$ cd falcon-sql-client
$ yarn install
$ yarn run rebuild:modules:electron
$ yarn run build
$ yarn run start

If, at some point, you don't find the time to complete a PR, please, let us know, so that someone else can continue the work.

Thank you for doing this!

@shannonlal
Copy link
Collaborator

@shannonlal shannonlal commented Jun 7, 2018

@julioasotodv If you have any issues with setting up the environment please let me know. I can jump a skype call and give you a hand

@n-riesco
Copy link
Collaborator

@n-riesco n-riesco commented Jul 6, 2018

@julioasotodv did you have a chance work on this feature? If you haven't, @kndungu is interested in implementing it.

@julioasotodv
Copy link
Author

@julioasotodv julioasotodv commented Jul 6, 2018

Hi, unfortunately I did not have any time at all :(

Also, I found out that my js is extremely rusty 😅

So yeah, @kndungu can go ahead anytime :)

@sidd1990
Copy link

@sidd1990 sidd1990 commented Jul 31, 2018

Hi all,

I am interested to contribute. I have a similar experience of developing the connector. But, I am new to open source contribution. Need help!

@shannonlal
Copy link
Collaborator

@shannonlal shannonlal commented Aug 1, 2018

@sidd1990 Which connector are you interested in adding. I can help you with the connector and UI part and walk you through the integration

@sidd1990
Copy link

@sidd1990 sidd1990 commented Sep 13, 2018

I am caught up with some personal work and I cannot get back to this for another month. But I will surely get back to it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.