Load Balance in Linux 2.6.32 - PowerPoint Presentation

401 views
Uploaded On 2016-05-09

Load Balance in Linux 2.6.32 - PPT Presentation

Load balancing Sung joon Choi RealTime Operating Systems Lab Seoul National University 20110915 Contents Load balancing Purpose Definition General cases Active load balancing Passive ID: 311704

run task load queue task run queue load core balance current softirq balancing idlest cases general linux function passive

Link:

Copy

Embed:

<iframe width="560" height="315" src="https://www.docslides.com/embed/311704" frameborder="0" allowfullscreen></iframe>

Download Presentation from below link

Download Presentation The PPT/PDF document "Load Balance in Linux 2.6.32" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript

Slide1

Load Balance in Linux 2.6.32 Load balancing

Sung-

joon

Choi

Real-Time Operating Systems Lab.

Seoul National University

2011-09-15Slide2

ContentsLoad balancingPurposeDefinitionGeneral casesActive load balancing

Passive

load balancingSpecial casesExecution of a new taskCPU’s shut down or intentionally being IDLELimitation

Load Balance in Linux 2.6.32Slide3

Load BalancingPurpose시스템에 코어 수보다 많은 수의 작업(task)이 있는 한, 모든 코어가

IDLE

상태

없이 수행하도록 조절Mechanism코어 간에 작업량 차이가 크지 않도록 조절DefinitionLoad balancing

SMP

구조에서 각 코어가 균등한 작업량

(load)을 가지도록 조절하는 것Load코어의 run-queue 가 갖는 모든 task 들의 weight 를 더한 값

Load Balance in Linux 2.6.32Slide4

Load BalancingDefinition (cont.)Idlest run-queueA run-queue that has the minimum load among the coresBusiest run-queueA run-queue that has the maximum value which is scale factor “load / (core’s power)”

모든

코어의 power 가 동일하다면 maximum load 를 갖는 코어의 run-queue 를 의미한다이종의 프로세서를 사용하는 시스템이라면 각 코어의

power

가 다를수도 있다

. 일반적으로 power는 capacity 또는 작업수행능력을 의미한다. Load Balance in Linux 2.6.32Slide5

ContentsLoad balancingPurposeDefinitionGeneral cases (mainly focused part)Active load balancing

Passive load balancing

Special cases

Execution of a new taskCPU’s shut down or intentionally being IDLELimitation

Load Balance in Linux 2.6.32Slide6

General

Cases

Active Load Balancing

Load Balance in Linux 2.6.32

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 4

Task 3

Task 5

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 4

Task 3

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 4

Task 3

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 3

READY

RUNNING

Going

to DEAD

Core

1 is going to IDLE

Run-queue is empty

Task migration

Task 2

(Assumption:

ll tasks have same weight)Slide7

Active Load BalancingImplementationWhen a task is going to end up its execution time do_exit

()

Sets task’s state to “

TASK_DEAD” schedule()In back-end procedure, if a core’s state is IDLE, it calls “idle_balance()”



idle_balance

()To pull a task on the busiest core’s run-queue, it calls “load_balance()” load_balance

()

Does a task migration

General

CasesSlide8

Active Load BalancingDrawbackActive load balancing 으로도 충분히 load balancing 을 달성할 수 있지만

코어 간 작업량 차이가 큰 상황인데도 각 태스크의 수행시간이 길어서

IDLE

상태를 갖게 되는 코어가 한동안 없다면, 단기간 내 load balancing 의 목적을 달성할 수 없다.이 상황을 피하기 위해서 주기적인 조절이 필요하다

General

CasesSlide9

General

Cases

Passive(Periodic)

load balancing

Load Balance in Linux 2.6.32

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 5

Task 4

Task 6

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 5

Task 3

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 5

Task 3

Run-queue

Current task

Core 0

Core 1

Task 4

Task 6

Task 3

Busiest run-queue

Idlest run-queue

Task 2

Task 3

For a long time, there is no IDLE core

Task 4

Task 6

Periodic check

If there

big

gap of load between

cores,

it is

uncomfortable

Task 4

Task 6

Task 1

Task 2

Task 5

Task migration

READY

RUNNING

Going

to DEAD

(Assumption:

ll tasks have same weight)Slide10

Passive Load BalancingTriggered by scheduler_tick()Tick value is compared with a parameter “next_balance” which is the time to do load balancing

Each run-queue has “

next_balance

”If a core takes the active load balancing, the parameter is set to 1 second afterIf a core takes the passive load balancing, the parameter is set to 1 minute after1초와 1분의 차이는 IDLE 상태를

밸런싱했던

코어는 다시

IDLE 상태가 되기 쉽기 때문에 곧바로 밸런싱을 해주기 위한 것Executed by bottom-half handlerA softirq named “SCHED_SOFTIRQ” is handled by “

run_rebalance_domains

()

”

General

CasesSlide11

Passive Load BalancingImplementation – start load balance General Cases

Timer interrupt invokes “

scheduler_tick

()

”

If the

tick value is equal to or greater than parameter

“

next_balance

”,

Busiest run-queue

Idlest run-queue

READY

RUNNING

(Assumption:

ll tasks have same weight)

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 5

Task 7

Task 3

Task 6

Next_balance

Task 4Slide12

Passive Load Balancing

Implementation – step1

General

Cases

If the

tick value is equal to or greater than parameter

“

next_balance

”,

Step1

: raises

softirq

“

SCHED_SOFTIRQ

” to kernel

Softirq

table

…

???

SCHED_SOFTIRQ

Busiest run-queue

Idlest run-queue

READY

RUNNING

(Assumption:

ll tasks have same weight)

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 5

Task 7

Task 3

Task 6

Next_balance

Task 4Slide13

Passive Load Balancing

Implementation – step2

General

Cases

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

ksoftirqd

Task 5

Task 7

Busiest run-queue

Idlest run-queue

Task 3

READY

RUNNING

(Assumption:

ll tasks have same weight)

If the

tick value is equal to or greater than parameter

“

next_balance

”,

Step1: raises a

softirq

“

SCHED_SOFTIRQ

” to kernel

Step2: finds the idlest run-queue to invoke a kernel thread “

ksoftirqd

”

Softirq

table

…

???

SCHED_SOFTIRQ

Task 6

Next_balance

Task 4Slide14

Passive Load Balancing

Implementation – step3

General

Cases

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

ksoftirqd

Task 5

Task 7

Busiest run-queue

Idlest run-queue

Task 3

READY

RUNNING

(Assumption:

ll tasks have same weight)

If the

tick value is equal to or greater than parameter

“

next_balance

”,

Step1: raises a

softirq

“

SCHED_SOFTIRQ

” to kernel

Step2: finds the idlest run-queue to invoke a kernel thread “

ksoftirqd

”

Softirq

table

…

???

SCHED_SOFTIRQ

Task 6

Next_balance

Task 4

Step3: the thread executes a function “

do_ksoftirqd

()

” that

picks a

softirq

and calls its handler function

run_rebalance_domains

()

Handler function (bottom-half handler)Slide15

Passive Load Balancing

Implementation – step4

General

Cases

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

ksoftirqd

Task 5

Task 7

Busiest run-queue

Idlest run-queue

Task 3

READY

RUNNING

(Assumption:

ll tasks have same weight)

If the

tick value is equal to or greater than parameter

“

next_balance

”,

Step1: raises a

softirq

“

SCHED_SOFTIRQ

” to kernel

Step2: finds the idlest run-queue to invoke a kernel thread “

ksoftirqd

”

Softirq

table

…

???

SCHED_SOFTIRQ

Task 6

Next_balance

Task 4

Step3: the thread executes a function “

do_ksoftirqd

()

” that

picks a

softirq

and calls its handler function

Step4:

the handler function finds the busiest run-queue to pull a task

run_rebalance_domains

()

Handler function (bottom-half handler)Slide16

Passive Load Balancing

Implementation – step5

General

Cases

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

ksoftirqd

Task 5

Task 7

Busiest run-queue

Idlest run-queue

Task 3

READY

RUNNING

(Assumption:

ll tasks have same weight)

If the

tick value is equal to or greater than parameter

“

next_balance

”,

Step1: raises a

softirq

“

SCHED_SOFTIRQ

” to kernel

Step2: finds the idlest run-queue to invoke a kernel thread “

ksoftirqd

”

Softirq

table

…

???

SCHED_SOFTIRQ

Task 6

Next_balance

Task 4

Step3: the thread executes a function “

do_ksoftirqd

()

” that

picks a

softirq

and calls its handler function

Step4:

the handler function finds the busiest run-queue to pull a task

Step5:

task migration

run_rebalance_domains

()

Handler function (bottom-half handler)

Task 4Slide17

Passive Load Balancing

Implementation

General

Cases

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 5

Task 6

Task 3

Task 4

Next_balance

Task 7

Run-queue

Current task

Core 0

Core 1

Task 1

Task 2

Task 5

Task 7

Task 3

Task 6

Next_balance

Task 4Slide18

Passive Load BalancingDrawbackThis algorithm has large overheadThe algorithm should check the maximum and minimum load out of all cores

And, if a current core is

not

the idlest one, The kernel thread “ksoftirqd” should be enqueued to the idlest run-queue of other core and waken upAlso, a current task of the target core that has the idlest run-queue is preempted by “ksoftirqd”

Tradeoff: balancing time interval



throughput latencyGeneral CasesSlide19

ContentsLoad balancingPurposeDefinitionGeneral casesActive load balancingPassive load balancing

Special cases

Execution of a new task

CPU’s shut down or intentionally being IDLELimitation

Load Balance in Linux 2.6.32Slide20

Special CasesExecution of a new taskWhen a new task is created in one core, kernel checks the core’s load whether it is reasonable to handle a new taskIf the load is unacceptable, current task of the core is migrated to the idlest core’s run-queue and rescheduledAnd a new task is executed in the core (not the idlest core)

CPU’s shut down or intentionally being IDLE

When one core should be shut down or intentionally be IDLE

, such as in POWER_SAVING_LOAD_BALANCEAll tasks in its run-queue are migrated to other coresActually, this case is just a task migrationLoad Balance in Linux 2.6.32Slide21

ContentsLoad balancingPurposeDefinitionGeneral casesActive load balancingPassive load balancing

Special cases

Execution of a new task

CPU’s shut down or intentionally being IDLELimitation

Load Balance in Linux 2.6.32Slide22

LimitationGlobal Fairness Global Fairness는 여러 개의 CPU로 이루어진 SMP

에서 모든

task

가 자신의 weight에 비례해서 run-time을 보장받는 정도를 의미한다. SMP 환경에서 Run queue가 CPU

에 하나씩 있고

, Load Balance

는 각 Run queue의 load(sum of weight)만을 고려해서 task를 옮기므로 task가 자신의 weight에 비례한 시간을 못 받는 경우가 생긴다.