[jira] [Assigned] (APEXCORE-119) Add Support For A New Type Of (Distributed) Operator

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[jira] [Assigned] (APEXCORE-119) Add Support For A New Type Of (Distributed) Operator

JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/APEXCORE-119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Timothy Farkas reassigned APEXCORE-119:
---------------------------------------

    Assignee:     (was: Timothy Farkas)

> Add Support For A New Type Of (Distributed) Operator
> ----------------------------------------------------
>
>                 Key: APEXCORE-119
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-119
>             Project: Apache Apex Core
>          Issue Type: Improvement
>            Reporter: Timothy Farkas
>
> This JIRA Proposes support for a new type of distributed operator. Currently when an operator is partitioned there is no platform supported mechanism through which partitions can talk to each other. A Distributed operator would have an easy to use platform supported mechanism through which operators in a partitioning can exchange information with each other. Eventually Distributed operators would support running plain old single threaded java code transparently across partitions.
> In summary the goals would be to do the following:
> 1 - provide a platform supported fault tolerant mechanism through which operators in a partitioning can talk to each other.
> 2 - provide a platform supported way to run plain old single threaded java code accross all the partitions of a Distributed operator
> The benefits of implementing this would be huge:
> 1 - Using distributed operators we could support large in memory fault tolerant data structures (graphs, maps, arrays) in a fault tolerant way. Like Spark's RDD's but better.
> 2 - Plain old java code could be used to access and manipulate the data structures, without the user having the learn complex API's like with Spark.
> An implementation proposal and presentation are coming soon.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Loading...