Skip to content

Example Extension Package needed #5

Description

@nicornk

Hi,

I am trying to understand the concepts behind the extensions for sparklyr.

An example extension would be very helpful for me.

Thanks!

Activity

  1. jjallaire commented on Jul 5, 2016

    @jjallaire
    Member

    We'll post a sample soon!

    On Tue, Jul 5, 2016 at 10:16 AM, Nicolas Rnkmp notifications@github.com
    wrote:

    Hi,

    I am trying to understand the concepts behind the extensions for sparklyr.

    An example extension would be very helpful for me.

    BR
    Nicolas

    —
    You are receiving this because you are subscribed to this thread.
    Reply to this email directly, view it on GitHub
    #5, or mute the thread
    https://github.com/notifications/unsubscribe/AAGXx1aRA_T0KPrj5uk7feeqPTaRYw-_ks5qSmcogaJpZM4JFKeP
    .

  2. jwijffels commented on Jul 20, 2016

    @jwijffels

    I tried this out and created an R package called spark.sas7bdat which imports SAS data in parallel in Spark
    More information: https://github.com/bnosac/spark.sas7bdat
    Thanks for the documentation about the usage of the sparkapi package. This worked nicely!

    One question. Is it desireable to include spark packages from https://spark-packages.org/ inside the java folder of the R package to reference it from there or is it advised to use the packages argument of the spark_dependency function.

  3. jjallaire commented on Jul 20, 2016

    @jjallaire
    Member

    That's fantastic!!!!

    Yes, you should definitely use the packages argument of the
    spark_dependency function (as you have done in your example) in preference
    to embedding within the java folder of the R package.

    On Wed, Jul 20, 2016 at 4:56 PM, jwijffels notifications@github.com wrote:

    I tried this out and created an R package called spark.sas7bdat which
    imports SAS data in parallel in Spark
    More information: https://github.com/bnosac/spark.sas7bdat
    Thanks for the documentation about the usage of the sparkapi package.
    This worked nicely!

    One question. Is it desireable to include spark packages from
    https://spark-packages.org/ inside the java folder of the R package to
    reference it from there or is it advised to use the packages argument of
    the spark_dependency function.

    —
    You are receiving this because you commented.
    Reply to this email directly, view it on GitHub
    #5 (comment), or mute
    the thread
    https://github.com/notifications/unsubscribe-auth/AAGXxxm7kZa9pX4LbhcUwILkBsN7qbsSks5qXouHgaJpZM4JFKeP
    .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions