Class responsibility Analysis of YARN NodeManager

标签：io   ar   os   sp   for   on   art   cti   ad   

When a container recieved a INIT_CONTAINER event, the state machine of containerImpl will execute the transaction RequestResourcesTransition, one of whose resposibility is to check if there is any resource requirement. If there is any, it will be transformed it to the LocalResourceRequest. It also gathers the resource with same LocalResource, then record the gathering into the containerImpl.

The brand new LocalResourceRequest will be added to the containerImpl‘s resource request records list. Then a Map with a visibility typed key and a LocalResourceRequest collection value is wrapped into a event called ContainerLocalizationRequestEvent. The event is passed to NM central event dispatcher then dispatch to ResourceLocalizationService.

ResourceLocalizationService
1.is a service;
2.is the access point of NM;
3.When recieves ContainerLocalizationRequestEvent, it create a file status cache firstly. Then create a LocalizerContext to define the basic information of this download, of cause the file status cache is in the LocalizerContext. This service will find a LocalResourceTracker to handle the LocalResourceR
equest. The LocalResourceTracker selection is based on the resource visibility, if the resource is a PUBLIC one, a LocalResourceTracker for public resou
rce will be selected, if the resource is in private level, a LocalResourceTracker is selected according to the user id, and if the resource is in an appliction level, a LocalResourceTracker is selected by applicatoin id.

LocalResourcesTracker
1. is a event handler;
2. maintains a map with LocalResourceRequest typed key, and LocalizedResource typed value;
3. When LocalResourceRequest is recieved, it will check whether the resource is not only existing in memory, but also exists in local disk. If the file is missing, it will decrease the counter and re-download the resource by passing the resource request to a localizedResource;

LocalizedResource
1. is a event handler with a state machine;
2. When recieved resource downloading request event, it account in itself, and send a LocalizerResourceRequestEvent message to LocalizerTracker

LocalizerTracker
1.is a service;
2.is a EventHandler;
3.process heartbeat from downloading threads;
4.Localizer has a PublicLocalizer and an group of LocalizerRunner;
5.when recieves LocalizerResourceRequestEvent, it will check the visibility of the event, if it is public, the event will be added to public localizer,
otherwise, will be add to localizerRunner only if the localizerRunner for this container(distinguished by container id) is not exist, then the locaizerr
unner is started. After that, the event will be passed to localizer.
6. When ContainerLocalizer is up, it will talk to LocaliserTracker by Heartbeat, but the heartbeat is pass to corresponding container id;

LocalizerRunner
1. A thread;
2. launch a Container executer to start up download threads;
3. process heartbeat to see whether there is pending tasks, if any, keep downloading the resources, all resource of a container is downloaded serially;

ContainerExecuter
1. launch the ContainerLocalizer to download local resource in blocking mode.

Class responsibility Analysis of YARN NodeManager

标签：io   ar   os   sp   for   on   art   cti   ad   

原文地址：http://www.cnblogs.com/masong/p/4162936.html