Sitelet https://github.com/trustedanalytics/atk/tree/master/engine-plugins/example-plugins
Skip to content

Latest commit

 

History

History

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 
To compile the project and create the sample jars do
 mvn package
you should see a example-plugins.jar in the target directory.

The hello world plugin will add a method to the frame class that prints hello world.
The word-count plugin will add a method to the frame class that counts words in the frame.
The vertex-degree count plugin will add a method to the graph class that counts the vertex out-degree.

To register the plugins with the Analytics Toolkit, update the following configuration files:
src/main/resources/atk-module.conf
src/main/resources/atk-plugin.conf

The 3 mentioned plugins are commented out in the default atk-plugin.conf file.
To enable them, uncomment those lines and rebuild.

Installing plugins:

To add the plugins to your current installation of trusted analytics copy the example-plugins.jar to
/usr/lib/trustedanalytics/lib/.  Plugins should go in a jar with "plugins" in the name of the jar.

Now restart the trusted analytics service and try the code example below.

sudo service trustedanalytics restart


Sample code:

from trustedanalytics import *

#the default home directory is  hdfs://user/atkuser all the sample data sets are saved to hdfs://user/atkuser/datasets
dataset = r"datasets/movie_data_random.csv"

#csv schema definition
schema = [("user_id", int32),
          ("movie_id", int32),
          ("rating", int32),
          ("splits", str)]

csv_file = CsvFile(dataset, schema, skip_header_lines=1)

print "Building data frame"

frame = Frame(csv_file)

print "View auto-generated Python documentation for plugin"
help(frame.helloworld)

print "Run hello-world plugin"
frame.helloworld()
>>u'Frame says Hello World'

frame.helloworld("test")
>> u'Frame says test'

More examples are available in the python-examples folder.