[jira] [Commented] (APEXCORE-10) Enable non-affinity of operators per node (not containers)

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[jira] [Commented] (APEXCORE-10) Enable non-affinity of operators per node (not containers)

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/APEXCORE-10?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15157939#comment-15157939 ]

ASF GitHub Bot commented on APEXCORE-10:
----------------------------------------

GitHub user ishark opened a pull request:

    https://github.com/apache/incubator-apex-core/pull/234

    (Only for Review) APEXCORE-10 Changes for supporting anti-affinity in operators

    Added list of affinity rules as attribute in DAGContext
    Added validation for affinity, anti-affinity and locality rules in dag validate phase
    Added handling of Container and Node affinity rules. Rack affinity is not supported at this point
    Also added Changes for supporting Node specific request in Cloudera
    Added Unit tests for affinity rules and dag validation for the same

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ishark/incubator-apex-core anti-affinity

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-apex-core/pull/234.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #234
   
----
commit 11c10d9dc885e783a35f1d79b204de80541e0879
Author: ishark <[hidden email]>
Date:   2016-02-04T01:06:52Z

    APEXCORE-10 Changes for supporting anti-affinity amongst operators
    Added list of affinity rules as attribute in DAGContext
    Added validation for affinity, anti-affinity and locality rules in dag validate phase
    Added handling of Container and Node affinity rules. Rack affinity is not supported at this point
    Also added Changes for supporting Node specific request in Cloudera
    Added Unit tests for affinity rules and dag validation for the same

----


> Enable non-affinity of operators per node (not containers)
> ----------------------------------------------------------
>
>                 Key: APEXCORE-10
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-10
>             Project: Apache Apex Core
>          Issue Type: Task
>            Reporter: Amol Kekre
>            Assignee: Isha Arkatkar
>              Labels: roadmap
>
> The issue happens on cloud which provides virtual cores with software like Xen underneath. In effect if CPU intensive operators land up on same node we have a resource bottleneck,
> Need to create an attribute that does the following
> - Operators A & B should not be on same node
> - Stram should use this attribute to try to get containers on different node
> It is understood that the user is making an explicit choice to use NIC instead of stream local optimization



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Loading...