Parent Directory
| 
Revision Log
Revision 1 - (view) (download) (as text)
| 1 : | mark | 1 | \documentclass{book} |
| 2 : | |||
| 3 : | \author{Oslo University College, Norway} | ||
| 4 : | \title{\sc High Volume Services Laboratory Handbook\\Version 0.1} | ||
| 5 : | \date{\today} | ||
| 6 : | |||
| 7 : | \usepackage{times} % -- good | ||
| 8 : | \usepackage{fancyheadings} | ||
| 9 : | \usepackage{tocloft} | ||
| 10 : | |||
| 11 : | %\usepackage{amsthm} | ||
| 12 : | \usepackage{a4} | ||
| 13 : | \usepackage{shadethm} | ||
| 14 : | \pagestyle{fancyplain} | ||
| 15 : | \lhead[\fancyplain{}{}]{\fancyplain{}{\rightmark}} | ||
| 16 : | \rhead[\fancyplain{}{\leftmark}]{\fancyplain{}{}} | ||
| 17 : | |||
| 18 : | |||
| 19 : | %\textheight 20cm | ||
| 20 : | \textwidth 15.5cm | ||
| 21 : | \oddsidemargin -20pt | ||
| 22 : | \evensidemargin -20pt | ||
| 23 : | |||
| 24 : | |||
| 25 : | %\topmargin -30pt | ||
| 26 : | \usepackage{url} | ||
| 27 : | \usepackage{tabularx} | ||
| 28 : | \usepackage{fancybox} | ||
| 29 : | \usepackage{fancyvrb} | ||
| 30 : | \usepackage{epsfig} | ||
| 31 : | %\usepackage{alltt} | ||
| 32 : | |||
| 33 : | |||
| 34 : | |||
| 35 : | \urlstyle{rm} | ||
| 36 : | %\def\UrlFont{\em} | ||
| 37 : | |||
| 38 : | |||
| 39 : | \newenvironment{code}% | ||
| 40 : | {\begin{Sbox}\begin{minipage}{.9\textwidth}\scriptsize}% | ||
| 41 : | {\end{minipage}\end{Sbox}% | ||
| 42 : | \begin{center}\setlength{\fboxsep}{2pt}\setlength{\fboxrule}{1pt}% | ||
| 43 : | \fbox{\TheSbox}\end{center}} | ||
| 44 : | |||
| 45 : | |||
| 46 : | \def\sex{\begin{alltt}\small} | ||
| 47 : | \def\esex{\end{alltt}} | ||
| 48 : | \def\ex{\begin{alltt}} | ||
| 49 : | \def\eex{\end{alltt}} | ||
| 50 : | |||
| 51 : | \def\file{\tt\small} | ||
| 52 : | |||
| 53 : | \def\samp{\tt\small} | ||
| 54 : | \def\kbd{\tt\small} | ||
| 55 : | \def\key{\sc} | ||
| 56 : | \def\var{\em} | ||
| 57 : | \def\beq{\begin{eqnarray}} | ||
| 58 : | \def\eeq{\end{eqnarray}} | ||
| 59 : | \def\2{\frac{1}{2}} | ||
| 60 : | |||
| 61 : | %\setcounter{tocdepth}{1} | ||
| 62 : | |||
| 63 : | |||
| 64 : | |||
| 65 : | \newshadetheorem{todo}{TO DO!} | ||
| 66 : | %\newshadetheorem{defin}{Definition} | ||
| 67 : | %\newtheorem*{corollary}{Corollary to principle} | ||
| 68 : | %\newshadetheorem{suggestion}{Suggestion} | ||
| 69 : | \newtheorem{example}{Example} | ||
| 70 : | \newtheorem{question}{Question} | ||
| 71 : | \makeindex | ||
| 72 : | |||
| 73 : | |||
| 74 : | \begin{document} | ||
| 75 : | %\begin{figure}[ht] | ||
| 76 : | %\psfig{file=NetlabCover.eps,width=14cm} | ||
| 77 : | %\end{figure} | ||
| 78 : | \maketitle | ||
| 79 : | \bibliographystyle{plain} | ||
| 80 : | |||
| 81 : | %\setlength{\cftchapnumwidth}{18pt} | ||
| 82 : | %\setlength{\cftsecnumwidth}{31pt} | ||
| 83 : | %\setlength{\cftsubsecnumwidth}{40pt} | ||
| 84 : | |||
| 85 : | \tableofcontents | ||
| 86 : | |||
| 87 : | |||
| 88 : | |||
| 89 : | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | ||
| 90 : | |||
| 91 : | \chapter{Introduction} | ||
| 92 : | |||
| 93 : | This is a working document. It will eventually be a handbook for a laboratory | ||
| 94 : | course in server room design. | ||
| 95 : | |||
| 96 : | Contributors to this booklet: Mark Burgess, Claudia Eriksen, Tore | ||
| 97 : | M. Jonassen Sven Ingebrigt Ulland, Gard Undheim. | ||
| 98 : | |||
| 99 : | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | ||
| 100 : | |||
| 101 : | \part{Data Centre Science} | ||
| 102 : | |||
| 103 : | \chapter{Safety and responsibility in the Lab} | ||
| 104 : | |||
| 105 : | |||
| 106 : | Oslo University College is committed to providing a safe and healthy | ||
| 107 : | working environment for its students and staff. You are expected to | ||
| 108 : | learn working practices that you will take with you when you leave | ||
| 109 : | the College. It is your ethical duty to spread safe working practices | ||
| 110 : | and procedures beyond this College in the workplace. | ||
| 111 : | |||
| 112 : | |||
| 113 : | Oslo University College requires its students, staff, and faculty to | ||
| 114 : | adhere to basic General Laboratory Safety Rules in the Electrical | ||
| 115 : | Engineering laboratories. | ||
| 116 : | |||
| 117 : | These rules apply to everyone. If you see someone ignoring the rules, | ||
| 118 : | it is your duty to remind them of their responsibilities. | ||
| 119 : | |||
| 120 : | \noindent {\em This discipline is a mark of professionalism -- it is not for | ||
| 121 : | annoyance or for show. You might find other labs that do not care about | ||
| 122 : | safety, but as a student of Oslo University College, we expect the highest | ||
| 123 : | standards.} | ||
| 124 : | |||
| 125 : | \section{General Laboratory Safety Rules} | ||
| 126 : | |||
| 127 : | \begin{enumerate} | ||
| 128 : | \item The lab is a dry and noisy environment. You should try to organize | ||
| 129 : | your work so that you do not spend more than three or four hours in the lab | ||
| 130 : | at any one time. If you need to talk to your work partners, go out of the | ||
| 131 : | lab into a quiet area rather than shouting. | ||
| 132 : | |||
| 133 : | \item Food and drink must not be brought into, stored in | ||
| 134 : | or consumed in a laboratory. | ||
| 135 : | |||
| 136 : | \item Never wear personal headphones or play music to drown out the noise: | ||
| 137 : | you can damage your hearing by playing loud music, and cause accidents by | ||
| 138 : | being distracted. Your full attention should be on your work. | ||
| 139 : | |||
| 140 : | \item Shoes must be worn, and appropriate | ||
| 141 : | personal clothing shall be worn in laboratories. | ||
| 142 : | |||
| 143 : | \item Do not wear items of metal jewelry when using electronic equipment. You might cause a short-circuit, get a shock or get the jewelry caught in something. | ||
| 144 : | |||
| 145 : | \item Bags and coats should be left at the doorway. | ||
| 146 : | |||
| 147 : | \item {\em BE TIDY AND TREAT ALL EQUIPMENT WITH RESPECT}. Do not tread on or run over cables with | ||
| 148 : | chairs or heavy objects. | ||
| 149 : | |||
| 150 : | \item Never drop items on the floor. Always put equipment in the correct box or storage place provided for it. Fold cables neatly to avoid long-term damage to them. | ||
| 151 : | |||
| 152 : | \item Learn and know what to do in an emergency. | ||
| 153 : | |||
| 154 : | \item Do not expose yourself or others to risk. Do not joke around in the lab. | ||
| 155 : | |||
| 156 : | \item Un-authorized persons MUST NOT be admitted to the laboratory. | ||
| 157 : | |||
| 158 : | 'Authorized' means having business in the laboratory with the permission | ||
| 159 : | of the College. No-one under the age of eighteen | ||
| 160 : | may enter the lab unsupervised. Only trained individuals and students enrolled in courses using the lab may touch the equipment. | ||
| 161 : | |||
| 162 : | \item Laboratory shall remain locked when unoccupied. | ||
| 163 : | \item Never open (remove cover) of any equipment in the laboratories, unless you are | ||
| 164 : | instructed to do so. | ||
| 165 : | \item Report all problems to drift@iu.hio.no (extensions 3268/3264/3263) | ||
| 166 : | \end{enumerate} | ||
| 167 : | |||
| 168 : | \section{Electrical Safety Guidelines} | ||
| 169 : | |||
| 170 : | \begin{enumerate} | ||
| 171 : | \item Be familiar with the electrical hazards associated with your workplace. | ||
| 172 : | |||
| 173 : | \item Be as careful for the safety of others as for yourself. Think before | ||
| 174 : | you act. Be tidy and systematic. | ||
| 175 : | \item Avoid bulky, loose or trailing clothes. Avoid long loose hair. Remove | ||
| 176 : | metal bracelets or watch-straps. | ||
| 177 : | |||
| 178 : | \item Do not take food or drink into the laboratory. Avoid wet | ||
| 179 : | hands and clothing. \item Use extension cords (skj\o | ||
| 180 : | teledninger) only when necessary and only on a temporary basis. | ||
| 181 : | Request new power outlets if your work requires equipment in an | ||
| 182 : | area without an outlet. | ||
| 183 : | |||
| 184 : | \item ALWAYS report damaged cords and cables, including cables that become hot, or with | ||
| 185 : | exposed wiring. Never ignore these or replace them for others to use. | ||
| 186 : | |||
| 187 : | |||
| 188 : | \item Experiments left unattended should be isolated. If | ||
| 189 : | for a special reason, it must be left on, a barrier and a warning notice | ||
| 190 : | are required. | ||
| 191 : | \item Equipment found to be faulty in any way should be reported immediately | ||
| 192 : | and not used until it is inspected and declared safe. | ||
| 193 : | |||
| 194 : | \item Voltages above 50 V RMS AC and 120 V DC are always dangerous. Extra precautions | ||
| 195 : | should be considered as voltage levels are increased. | ||
| 196 : | |||
| 197 : | \item Never make any changes to circuits or mechanical layout without first | ||
| 198 : | isolating the circuit by switching off and removing connections to supplies. | ||
| 199 : | |||
| 200 : | \item Know what you must do in an emergency. | ||
| 201 : | |||
| 202 : | \item Be acquainted with the Emergency Power Off: | ||
| 203 : | |||
| 204 : | |||
| 205 : | \item When touching equipment that might be electrically "live" always use the {\em back of your hand}. Never grip anything with your fingers, since and electric shock will tend to make you grip tighter. | ||
| 206 : | |||
| 207 : | \item Electrical power sockets should not be overloaded. | ||
| 208 : | |||
| 209 : | \item {\b No cables should ever be stretched.} Tape them firmly to a table or floor. Remember that people can trip over cables. | ||
| 210 : | |||
| 211 : | \item Hold the plug firmly by the insulating cover when plugging it in or unplugging it. | ||
| 212 : | {\bf NEVER} unplug an electrical device by pulling on its cord! | ||
| 213 : | |||
| 214 : | \item Do not attempt to lift or move heavy equipment alone. | ||
| 215 : | |||
| 216 : | \end{enumerate} | ||
| 217 : | |||
| 218 : | \section{Electrical Emergency Response} | ||
| 219 : | |||
| 220 : | The following instructions provide guidelines for handling two types | ||
| 221 : | of electrical emergencies: | ||
| 222 : | |||
| 223 : | |||
| 224 : | When someone suffers serious electrical shock, he or she may be | ||
| 225 : | knocked unconscious. If the victim is still in contact with the | ||
| 226 : | electrical current, immediately turn off the electrical power | ||
| 227 : | source. If you cannot disconnect the power source, push in the | ||
| 228 : | Emergency Power Off button. | ||
| 229 : | \begin{enumerate} | ||
| 230 : | |||
| 231 : | \item Do not touch a victim that is still in contact with a power source; you | ||
| 232 : | could electrocute yourself. | ||
| 233 : | |||
| 234 : | \item Have someone call for emergency medical assistance immediately. Administer | ||
| 235 : | first-aid, as appropriate. | ||
| 236 : | |||
| 237 : | \item If an electrical fire occurs, try to disconnect the electrical power | ||
| 238 : | source, if possible. | ||
| 239 : | |||
| 240 : | |||
| 241 : | |||
| 242 : | \item In case of fire, pull out power sockets if possible, taking care not to be electrocuted. | ||
| 243 : | \item Use of {\bf clean water} to put out electrical fire is acceptable. | ||
| 244 : | \end{enumerate} | ||
| 245 : | |||
| 246 : | |||
| 247 : | \section{Handling equipment} | ||
| 248 : | |||
| 249 : | |||
| 250 : | All electronic equipment should be treated as highly fragile and | ||
| 251 : | easily damaged, regardless of how sturdy it is. Today we are far too | ||
| 252 : | blas\'e towards electronic equipment. | ||
| 253 : | \begin{itemize} | ||
| 254 : | \item Never insert or remove power cords to equipment without ensuring that it is switched off. | ||
| 255 : | \item Take care when inserting multi-pin connectors that the pins are oriented the right way up and that no pins are bent on insertion. | ||
| 256 : | \end{itemize} | ||
| 257 : | Moreover: | ||
| 258 : | \begin{itemize} | ||
| 259 : | \item {\em Read instructions:} | ||
| 260 : | When dealing with hardware, one should always look for and {\em read} | ||
| 261 : | instructions in a manual. It is foolish to make assumptions about expensive | ||
| 262 : | purchases. Instructions are there for a reason. | ||
| 263 : | \item {\em Interfaces and connectors:} | ||
| 264 : | Hardware is often connected to an interface by a cable or connector. | ||
| 265 : | Obtaining the correct cable is of vital importance. Many manufacturers | ||
| 266 : | use cables which look similar, superficially, but which actually are | ||
| 267 : | different. An incorrect cable can result in damage to an interface. | ||
| 268 : | Modem cables in particular can damage a computer or modem if they | ||
| 269 : | are incorrectly wired, since some computers supply power through these | ||
| 270 : | cables which can damage equipment which does not expect to find a | ||
| 271 : | power supply coming across the cable. | ||
| 272 : | |||
| 273 : | \item {\em Handling components:} | ||
| 274 : | Modern day CMOS chips work at low voltages (typically 5 volts or lower). | ||
| 275 : | Standing on the floor with insulating shoes, you can pick up a | ||
| 276 : | static electric charge of several thousand volts. Such a charge | ||
| 277 : | can instantly destroy computer chips. Before touching any | ||
| 278 : | computer components, earth yourself by touching the metal | ||
| 279 : | casing of the computer. If you are installing equipment inside | ||
| 280 : | a computer, wear a conductive wrist strap. Avoid wearing rubber sandals | ||
| 281 : | or shoes that insulate you from Earth when dealing with | ||
| 282 : | open case equipment, since these cause the body to | ||
| 283 : | build up charge that can discharge through that equipment; on the | ||
| 284 : | other hand this might be good advice when working around high voltage | ||
| 285 : | or current sources. | ||
| 286 : | \index{Components, handling} | ||
| 287 : | \index{Handling components} | ||
| 288 : | |||
| 289 : | \item {\em Disks:} | ||
| 290 : | Disk technology has been improving steadily for two decades. The most | ||
| 291 : | common disk types, in the workplace, fall into two families: ATA | ||
| 292 : | (formerly IDE) and SCSI. The original IDE (Integrated Drive | ||
| 293 : | Electronics) and SCSI (Small Computer Software Interface) had | ||
| 294 : | properties that have since evolved faster than the prejudices about | ||
| 295 : | them. ATA (formerly IDE) disks are now generally cheaper than SCSI | ||
| 296 : | disks (due to volume sales) and excel at sequential access, but SCSI | ||
| 297 : | disks have traditionally been more efficient at handling multiple | ||
| 298 : | accesses due to a multitasking bus design, and are therefore better in | ||
| 299 : | multitasking systems, where random access is important. However, | ||
| 300 : | filesystem design also plays an important role in determining the | ||
| 301 : | perceived performance of each; i.e. how operating systems utilize | ||
| 302 : | buses during updates is at least as important as bus performance | ||
| 303 : | itself. Interesting comparisons show that IDE technology has caught up | ||
| 304 : | with the head start that SCSI disks once had\cite{white1} for many | ||
| 305 : | purposes, but not all. | ||
| 306 : | |||
| 307 : | SCSI\cite{lisa93157} comes in several varieties, SCSI 1, SCSI | ||
| 308 : | 2, wide SCSI, fast-wide etc. The difference has to do with the width | ||
| 309 : | of the data-bus and the number of disks which can be attached to each | ||
| 310 : | controller. There are presently three SCSI standards SCSI-1, SCSI-2 | ||
| 311 : | and SCSI-3. The SCSI-2 standard defines also wide, fast and fast/wide | ||
| 312 : | SCSI. | ||
| 313 : | Each SCSI disk has its own address (or number) which must be set by | ||
| 314 : | changing a setting on the disk-cabinet or by changing jumper settings | ||
| 315 : | inside the cabinet. Newer disks have programmable identities. | ||
| 316 : | \index{SCSI disks} | ||
| 317 : | \index{IDE disks} | ||
| 318 : | \index{ATA disks} | ||
| 319 : | Disk chain buses must be terminated with a proper terminating | ||
| 320 : | connector. Newer disks often contain automatic termination mechanisms | ||
| 321 : | integrated into the hardware. | ||
| 322 : | The devices on the SCSI bus talk to the computer through a | ||
| 323 : | controller. On modern PCs the SCSI controller is usually connected to | ||
| 324 : | the PCI bus either as an on-board solution on motherboards or as a | ||
| 325 : | separate card in a PCI slot. Other buses are also used as the carrier | ||
| 326 : | of the SCSI protocol, like FireWire (IEEE 1394) and USB. The SCSI | ||
| 327 : | standard also supports removable media devices (CD-ROM, CD-R, Zip | ||
| 328 : | drives), video frame grabbers, scanners and tape streamers (DAT, DLT). | ||
| 329 : | |||
| 330 : | |||
| 331 : | \item {\em Memory:} | ||
| 332 : | Memory chips are sold on small pluggable boards. They | ||
| 333 : | are sold in different sizes and with different speeds. | ||
| 334 : | A computer has a number of slots where they can be installed. | ||
| 335 : | When buying and installing RAM, remember | ||
| 336 : | \index{SIMM} | ||
| 337 : | \begin{itemize} | ||
| 338 : | |||
| 339 : | \item The physical size of memory plug-ins is important. Not all of | ||
| 340 : | them fit into all sockets. | ||
| 341 : | |||
| 342 : | \item Memory is sold in units with different capacities and data rates. One | ||
| 343 : | must find out what size can be used in a system. In many cases one | ||
| 344 : | may not mix different types. | ||
| 345 : | |||
| 346 : | \item There are various incompatible kinds of RAM that work in | ||
| 347 : | different ways. Error correcting RAM, for instance, is tolerant to | ||
| 348 : | error from external noise sources like cosmic rays and other ultra | ||
| 349 : | short wave disturbances. It is recommended for important servers, | ||
| 350 : | where stability is paramount. | ||
| 351 : | |||
| 352 : | \item On some computers one must fill up RAM slots in a particular | ||
| 353 : | order, otherwise the system will not be able to find them. | ||
| 354 : | \end{itemize} | ||
| 355 : | |||
| 356 : | \end{itemize} | ||
| 357 : | |||
| 358 : | |||
| 359 : | \index{Weather} | ||
| 360 : | Another aspect of hardware is the extent to which weather and | ||
| 361 : | environment are important for operation.. | ||
| 362 : | |||
| 363 : | \begin{itemize} | ||
| 364 : | |||
| 365 : | \item {\em Lightning:} | ||
| 366 : | strikes can destroy fragile equipment. No fuse will protect hardware | ||
| 367 : | from a lightning strike. Transistors and CMOS chips burn out much faster | ||
| 368 : | than any fuse. Electronic spike protectors can help here, but nothing | ||
| 369 : | will protect against a direct strike. | ||
| 370 : | |||
| 371 : | \item {\em Power:} | ||
| 372 : | failure can cause disk damage and loss of data. A UPS | ||
| 373 : | (Un-interruptible power supply) can help. | ||
| 374 : | |||
| 375 : | \item {\em Heat:} | ||
| 376 : | The blazing summer heat or a poorly placed heater oven can cause | ||
| 377 : | systems to overheat and suddenly black out. One should not let the | ||
| 378 : | ambient temperature near a computer to rise much above about 25 | ||
| 379 : | degrees Centigrade. Clearly some equipment can tolerate heat, better | ||
| 380 : | than other equipment. Bear in mind that metals expand significantly, | ||
| 381 : | so moving parts like disks will be worst affected by heat. Increased | ||
| 382 : | temperature also increases noise levels that can reduce network | ||
| 383 : | capacities by a fraction of a percent. While this might not sound like | ||
| 384 : | much, a fraction of a percent of a Giga-bit cable is a lot of | ||
| 385 : | capacity. Heat can cause RAM to operate unpredictably and disks to | ||
| 386 : | misread/miswrite. Good ventilation is essential for computers and | ||
| 387 : | screens for avoiding electrical faults. | ||
| 388 : | |||
| 389 : | \item {\em Cold:} | ||
| 390 : | Sudden changes from hot to cold are just as bad. They can | ||
| 391 : | cause unpredictable changes in electrical properties of chips and | ||
| 392 : | cause systems to crash. In the long term, these changes could lead | ||
| 393 : | to cracks in the circuit boards and irreparable chip damage. | ||
| 394 : | |||
| 395 : | \item {\em Humidity:} In times of very cold weather and very | ||
| 396 : | dry heat, the humidity falls to very low levels. At these times, the | ||
| 397 : | amount of static electricity builds up to quite high levels, without | ||
| 398 : | dissipating. This can be a risk to electronic circuitry. Humans pick | ||
| 399 : | up charge just by walking around, that can destroy fragile | ||
| 400 : | circuitry. Paper sticks together causing paper crashes in laser | ||
| 401 : | printers. Too much humidity can lead to condensation and short circuits. | ||
| 402 : | |||
| 403 : | \end{itemize} | ||
| 404 : | How could you go about verifying that conditions in the lab are | ||
| 405 : | appropriate? | ||
| 406 : | |||
| 407 : | |||
| 408 : | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | ||
| 409 : | |||
| 410 : | |||
| 411 : | |||
| 412 : | \chapter{Power in the lab} | ||
| 413 : | |||
| 414 : | Power consumption is defined as the flow of energy, per unit time, in | ||
| 415 : | or out of a system. It is measured in Joules per second, or Watts or | ||
| 416 : | Volt-Amperes, which are all equivalent. | ||
| 417 : | |||
| 418 : | Electrical power is needed to make computers run. It is also needed | ||
| 419 : | for lighting and cooling. Every electrical device generates heat, including | ||
| 420 : | cooling equipment! Once you have estimated your power needs for computers, | ||
| 421 : | you should add about 70\% again for the cost of cooling. | ||
| 422 : | |||
| 423 : | |||
| 424 : | You should acquaint yourself with the power system in our local country. | ||
| 425 : | Look at the excellent information on Wikepedia about international mains supplies. | ||
| 426 : | \begin{itemize} | ||
| 427 : | \item http://en.wikipedia.org/wiki/Mains\_electricity | ||
| 428 : | |||
| 429 : | \item http://en.wikipedia.org/wiki/Domestic\_AC\_power\_plugs\_and\_sockets | ||
| 430 : | |||
| 431 : | \item http://en.wikipedia.org/wiki/List\_of\_countries\_with\_mains\_power\_plugs,\_voltages\_and\_frequencies | ||
| 432 : | \end{itemize} | ||
| 433 : | |||
| 434 : | The following text is from Wikepedia: | ||
| 435 : | \bigskip | ||
| 436 : | |||
| 437 : | \framebox{ | ||
| 438 : | \begin{minipage}[t]{0.8\linewidth} | ||
| 439 : | |||
| 440 : | {\em European and most other countries use a supply that is within 10\% of | ||
| 441 : | 230 V, whereas Japan and most of the Americas use between 100 and 127 | ||
| 442 : | V. Following voltage harmonisation co-ordinated with CENELEC | ||
| 443 : | countries, all electricity supply within the EU is now nominally 230 V | ||
| 444 : | +/− 10\% (though some countries have stricter specifications: for | ||
| 445 : | example, the UK specifies 230 V +10\% -6\%). In practice this means that | ||
| 446 : | countries such as the UK that previously supplied 240 V continue to do | ||
| 447 : | so, and those that previously supplied 220 V continue to do | ||
| 448 : | so. However equipment should be designed to accept any voltages within | ||
| 449 : | the specified range, and in practice most do so. | ||
| 450 : | Similarly, Australia | ||
| 451 : | has converted to 230 V as the nominal standard, and like the UK, 240 V | ||
| 452 : | is within the allowable tolerance. "240 volt" spoken as "two forty | ||
| 453 : | volt" remains a synonym for mains in Australian and British English. | ||
| 454 : | |||
| 455 : | |||
| 456 : | In the United States and Canada, mains power is referred to by a | ||
| 457 : | variety of names, including household power (or household | ||
| 458 : | electricity), domestic power, wall power (because it comes from a | ||
| 459 : | power outlet in the wall), or simply "AC power." A close synonym for | ||
| 460 : | "mains" in Canadian provinces that use hydroelectric power is "hydro". | ||
| 461 : | ANSI standard C84.1 and Canadian standard CAN3-C235 specify that the | ||
| 462 : | nominal voltage at the output should be 120 V and allow a range of 114 | ||
| 463 : | to 126 V. | ||
| 464 : | |||
| 465 : | In Japan, the electrical power supply to households is at | ||
| 466 : | 100 V. Eastern and northern parts of Honshu (including Tokyo) and | ||
| 467 : | Hokkaido have a frequency of 50 Hz, whereas western Honshu (including | ||
| 468 : | Nagoya, Osaka, and Hiroshima), Shikoku, Kyushu and Okinawa operate at | ||
| 469 : | 60 Hz. To accommodate the difference, appliances marketed in Japan can | ||
| 470 : | often be switched between the two frequencies.} | ||
| 471 : | |||
| 472 : | \end{minipage} | ||
| 473 : | } | ||
| 474 : | |||
| 475 : | |||
| 476 : | \section{Power Interference: wireless signals and computer power supplies} | ||
| 477 : | |||
| 478 : | \begin{itemize} | ||
| 479 : | \item How much electrical power does a computer use? | ||
| 480 : | |||
| 481 : | \item Does it use the same amount of power in your house as it does in a data centre? | ||
| 482 : | \end{itemize} | ||
| 483 : | |||
| 484 : | To answer these questions, we need to know some basic physics of waves | ||
| 485 : | - because A.C. power is a wave, and energy is carried in the wave-like | ||
| 486 : | motion of the electrons of the wires. Recall your basic networking: | ||
| 487 : | the same principles of energy transfer apply to the alternating currents feeding | ||
| 488 : | computer power supplies as to the absorption of wireless signals in | ||
| 489 : | walls. We need to understand how much of a wave is absorbed and does | ||
| 490 : | work (heating up the wall or driving and heating up the CPU) and how | ||
| 491 : | much simply passes through or is refleected back again. | ||
| 492 : | |||
| 493 : | Electronic circuitry has two kinds of components: | ||
| 494 : | \begin{itemize} | ||
| 495 : | |||
| 496 : | \item {\em Resistive components} absorb and dissipate power as heat. They obey Ohm's law. | ||
| 497 : | |||
| 498 : | \item {\em Reactive components} absorb and return energy (like a battery), by storing | ||
| 499 : | it in electromagnetic fields. They include capacitors (condensers) | ||
| 500 : | and inductors (electro-magnetic coils). | ||
| 501 : | \end{itemize} | ||
| 502 : | |||
| 503 : | Reactive components (inductors and capacitors) store and return most | ||
| 504 : | of the power they use. This means that if we wait for a full cycle of the | ||
| 505 : | alternating wave, we will get back most of what we put in again. Resistive | ||
| 506 : | components, on the other hand, convert most of the energy going into them into | ||
| 507 : | heat. | ||
| 508 : | |||
| 509 : | What this means is that, if we measure the absorption of energy (the | ||
| 510 : | power consumption) of a device we must follow its progress over many | ||
| 511 : | cycles and take an average to see how much is being absorbed forever, | ||
| 512 : | and how much is being borrowed as a short-term loan and returned. In | ||
| 513 : | other words, we are interested in the {\em trend}, not the | ||
| 514 : | fluctuations of energy transfer. | ||
| 515 : | |||
| 516 : | \subsection{Power in DC circuits} | ||
| 517 : | |||
| 518 : | In direct current (DC) circuits, the power (energy released per unit time) is given by | ||
| 519 : | \beq | ||
| 520 : | P = IV | ||
| 521 : | \eeq | ||
| 522 : | where $I$ is the constant current and $V$ is the constant voltage. In | ||
| 523 : | a device where a DC current can flow, reactive components to not have | ||
| 524 : | an effect and Ohm's law applies: | ||
| 525 : | \beq | ||
| 526 : | V = IR | ||
| 527 : | \eeq | ||
| 528 : | where $R$ is the electrical resistance. In this case, we can write | ||
| 529 : | \beq | ||
| 530 : | P = IV = I^2R = V^2/R. | ||
| 531 : | \eeq | ||
| 532 : | There is no analogue of a DC circuit for an electromagnetic wave. | ||
| 533 : | |||
| 534 : | \subsection{AC circuits: `clean' power} | ||
| 535 : | |||
| 536 : | For an alternating current (wave) source, there are several cases to | ||
| 537 : | consider. We start with the simplest. A crucial point here is that | ||
| 538 : | Ohm's law is no longer true in AC circuits, because capacitors and | ||
| 539 : | inductors can borrow power for a short time and then return it, like a | ||
| 540 : | wave interacting with a dielectric. The voltage and current are now functions | ||
| 541 : | of time: | ||
| 542 : | \beq | ||
| 543 : | I &=& I(t)\nonumber\\ | ||
| 544 : | V &=& V(t). | ||
| 545 : | \eeq | ||
| 546 : | |||
| 547 : | It is still true that the instantaneous power consumption (the total energy | ||
| 548 : | absorbed per unit time) is | ||
| 549 : | \beq | ||
| 550 : | P = IV = I(t)V(t). | ||
| 551 : | \eeq | ||
| 552 : | However, mains power is tyically varying at 50-60 Hz, so this is not | ||
| 553 : | an true reflection of the long term behaviour, only what happens on | ||
| 554 : | the scale of tenths of a second. | ||
| 555 : | |||
| 556 : | If we consider a system driven by a single frequency (clean) wave, the short | ||
| 557 : | term borrowing of current is reflected by a phase shift between the | ||
| 558 : | voltage, which is the driving force of a systems, and the current, | ||
| 559 : | which is the response of the system to the driving force. Let us call | ||
| 560 : | this phase shift $\phi$. | ||
| 561 : | \beq | ||
| 562 : | V(t) &=& V_0 \sin (2\pi f t)\nonumber\\ | ||
| 563 : | I(t) &=& I_0 \sin (2\pi f t+\phi) | ||
| 564 : | \eeq | ||
| 565 : | where $f$ frequency and $T = 1/f$ is the period of the wave. | ||
| 566 : | |||
| 567 : | |||
| 568 : | \bigskip | ||
| 569 : | \framebox{ | ||
| 570 : | \begin{minipage}[t]{0.8\linewidth} | ||
| 571 : | |||
| 572 : | It helps to think about voltage and current in electrical circuits in terms | ||
| 573 : | of liquid flow. The voltage is like an external pressure that wants to make the | ||
| 574 : | current flow through a pipe (wire). When we say that there is a phase delay | ||
| 575 : | between current and voltage, we are saying that current and voltage are not | ||
| 576 : | moving exactly in synch'. | ||
| 577 : | |||
| 578 : | Why would current not start flowing at the same time as the driving | ||
| 579 : | pressure? There must be some inertia in the system for this to happen. | ||
| 580 : | In a constant current (D.C.) we will not notice this inertia, because | ||
| 581 : | it is constant. But if we change the voltage at high speed, the | ||
| 582 : | inertia becomes more and more noticable. You can imagine trying to saw | ||
| 583 : | a piece of wood while you are standing on a raft, on choppy water. As | ||
| 584 : | you try to saw a piece of wood, motion the waves underneath the raft | ||
| 585 : | interferes with your sawing. Sometimes, pushing while you are pulling | ||
| 586 : | etc. Thus your ability to transfer energy to the wood depends on the | ||
| 587 : | phase between your sawing and the waves. | ||
| 588 : | |||
| 589 : | \end{minipage} | ||
| 590 : | } | ||
| 591 : | \bigskip | ||
| 592 : | |||
| 593 : | We can compute the average power over a number of cycles $nT$ by | ||
| 594 : | computing | ||
| 595 : | \beq | ||
| 596 : | \langle P\rangle_\phi = \frac{1}{nT} \int_0^{nT} \; I(t)\,V(t)\, dt | ||
| 597 : | \eeq | ||
| 598 : | We can evaluate this using two trigonometric identities: | ||
| 599 : | \beq | ||
| 600 : | \sin(A+B) &=& \sin A \cos B + \cos A \sin B\\ | ||
| 601 : | \sin^2 X &=& \2(1-\cos 2X) | ||
| 602 : | \eeq | ||
| 603 : | Using the first of these to rewrite $I(t)$, we have | ||
| 604 : | \beq | ||
| 605 : | \langle P\rangle &=& \frac{I_0V_0}{nT}\int_0^{nT} \left[ | ||
| 606 : | \sin^2\left(\frac{2\pi t}{T}\right)\cos\phi + | ||
| 607 : | \sin\left(\frac{2\pi t}{T}\right)\cos\left(\frac{2\pi t}{T}\right)\sin\phi | ||
| 608 : | \right] dt | ||
| 609 : | \eeq | ||
| 610 : | Rewriting the first term with the help of the second identity allows us to | ||
| 611 : | integrate the expression. Most terms vanish showing the power that | ||
| 612 : | is returned over a whole cycle. We are left with: | ||
| 613 : | \beq | ||
| 614 : | \langle P\rangle_\phi &=& \2I_0V_0\cos\phi. | ||
| 615 : | \eeq | ||
| 616 : | In terms of the normally quoted root-mean-square (RMS) values: | ||
| 617 : | \beq | ||
| 618 : | V_{\rm rms} = \sqrt{\frac{1}{nT}\int_0^{nT} (V(t))^2} | ||
| 619 : | \eeq | ||
| 620 : | For a mono-chromatic sine wave, one has simply: | ||
| 621 : | \beq | ||
| 622 : | V_{\rm rms} &=& V_0/\sqrt{2}\nonumber\\ | ||
| 623 : | I_{\rm rms} &=& I_0/\sqrt{2}, | ||
| 624 : | \eeq | ||
| 625 : | we have | ||
| 626 : | \beq | ||
| 627 : | \langle P\rangle_\phi &=& I_{\rm rms}V_{\rm rms}\cos\phi. | ||
| 628 : | \eeq | ||
| 629 : | The RMS values are the values returned by most measuring devices and | ||
| 630 : | they are the values quoted on power supplies. | ||
| 631 : | The cosine factor is sometimes called the ``power factor'' in | ||
| 632 : | electrical engineering literature. It is generalized below. Clearly | ||
| 633 : | \beq | ||
| 634 : | \langle P\rangle_{\phi>0} \le \langle P\rangle_{\phi=0}. | ||
| 635 : | \eeq | ||
| 636 : | In other words, the actual power consumption is not necessarily | ||
| 637 : | as bad as the values one would measure with volt and ammeters. | ||
| 638 : | This is a pity when we pay our bill, because the power companies | ||
| 639 : | measure current, not work-done. That means we typically pay more than | ||
| 640 : | we should for electrical power. | ||
| 641 : | |||
| 642 : | |||
| 643 : | \subsection{AC circuits, clipped and dirty power} | ||
| 644 : | |||
| 645 : | Not all power is transferred by a clean harmonic signal like the one | ||
| 646 : | above. Anything that is connected to power with a reactive load | ||
| 647 : | (capacitors and induction coils) sends power back into the net and | ||
| 648 : | adds noise to the regular 50-60 Hz. Moreover, an inability to deliver | ||
| 649 : | enough current can leave to clipping of the wave form, which generates | ||
| 650 : | Fourier harmonics. Switched mode power supplies in personal computers, | ||
| 651 : | and flourescent lighting which is highly reactive can add harmonics to | ||
| 652 : | the power supply. For that reason, the actual phase difference is not | ||
| 653 : | just a function of your device, but of all the devices close to your | ||
| 654 : | device. The result can be inefficient power transfer. | ||
| 655 : | |||
| 656 : | |||
| 657 : | A direct calculation of the reduction in power transfer is impractical | ||
| 658 : | in this case, and one simply defined | ||
| 659 : | \beq | ||
| 660 : | PF = \cos\phi \equiv \frac{\langle P\rangle}{ I_{\rm rms}V_{\rm rms}} | ||
| 661 : | \eeq | ||
| 662 : | |||
| 663 : | More expensive equipment is designed with `power factor corrected' | ||
| 664 : | electronics that attempts to reduce the reactance of the load and lead | ||
| 665 : | to a simple $\phi=0$ behaviour. The power factor must be measured in | ||
| 666 : | situ to truly know what it is. | ||
| 667 : | \begin{itemize} | ||
| 668 : | |||
| 669 : | \item Some authors claim that power factors | ||
| 670 : | of $\cos\phi = 0.6$ have been observed in some PC hardware. | ||
| 671 : | |||
| 672 : | \item Some authors suggest that a mean value of $\cos\phi = 0.8$ is a reasoable | ||
| 673 : | guess. | ||
| 674 : | |||
| 675 : | \item Others claim that in data centres one should assume that $\cos\phi=1$ to | ||
| 676 : | correctly allow for enough headroom to deal with power demand. | ||
| 677 : | \end{itemize} | ||
| 678 : | |||
| 679 : | Another side effect of mutliple harmonics is that the RMS value is not | ||
| 680 : | simply $1/\sqrt{2} = 1/1.4$ of the peak value (amplitude). This leads to a new ratio | ||
| 681 : | called the ``crest factor'', which is simply: | ||
| 682 : | \beq | ||
| 683 : | {\rm Crest} = \frac{I_0}{I_{\rm rms}}. | ||
| 684 : | \eeq | ||
| 685 : | |||
| 686 : | |||
| 687 : | \begin{itemize} | ||
| 688 : | |||
| 689 : | \item For a clean sinusoidal wave, this ratio is simply $\sqrt 2 =1.4$. | ||
| 690 : | |||
| 691 : | \item Some authors claim that the crest factor can be as high as 2-3 for cheap | ||
| 692 : | computing equipment. | ||
| 693 : | |||
| 694 : | \item Some authors claim that a value of 1.4-1.9 is appropriate for power delivered by a UPS | ||
| 695 : | (uninterruptible power supply). | ||
| 696 : | |||
| 697 : | \item The crest factor is load dependent. | ||
| 698 : | \end{itemize} | ||
| 699 : | |||
| 700 : | |||
| 701 : | \subsection{Some jargon} | ||
| 702 : | |||
| 703 : | Based on the background above, one sees the following terminology in use, | ||
| 704 : | often in connection with uninterruptible power supplies. | ||
| 705 : | \begin{itemize} | ||
| 706 : | |||
| 707 : | \item $I_{\rm rms}V_{\rm rms}$ is referred to as ``VA'' or Volt-Amperes, not ``Watts''. | ||
| 708 : | |||
| 709 : | \item Power consumption is referred to as VA $\times$ PF or Volt-Amperes multiplied | ||
| 710 : | by the power factor and is measured in ``Watts''. | ||
| 711 : | |||
| 712 : | \end{itemize} | ||
| 713 : | |||
| 714 : | |||
| 715 : | \section{Earth/Ground} | ||
| 716 : | |||
| 717 : | Current flows when there is a difference in electrical voltage across a closed | ||
| 718 : | circuit. It flows from regions of high voltage to regions of low voltage, | ||
| 719 : | i.e. it is differences, not absolutes that matter. | ||
| 720 : | |||
| 721 : | The Earth or ground literally means the electrical potential of the | ||
| 722 : | ground beneath a building. This has normally a lower potential than | ||
| 723 : | anything else and so it is used as a safety device to prevent electric | ||
| 724 : | shocks. | ||
| 725 : | |||
| 726 : | The idea is that, anything metallic can condiuct electricity. In the | ||
| 727 : | event of a fault, a metallic box or rack could obtain a high | ||
| 728 : | voltage. If a person touches this, while standing on the ground, there | ||
| 729 : | will be a path through that person to Earth and the person will | ||
| 730 : | receive an electric shock. | ||
| 731 : | |||
| 732 : | If, on the other hand, the metal casing is connected to Earth directly | ||
| 733 : | by a wire, most of the current will flow through the wire to Earth, | ||
| 734 : | not through the person, because it is a much better conductor -- hence | ||
| 735 : | the person will be protected. (There is no point is Earthing/grounding | ||
| 736 : | a device that has no external metallic parts.) | ||
| 737 : | |||
| 738 : | This theory is simple and striaghtforward, but it has a number of | ||
| 739 : | other consequences. | ||
| 740 : | \begin{enumerate} | ||
| 741 : | \item The voltage/potential of Earth is not constant everywhere. | ||
| 742 : | This means that currents can flow through the Earthing system of a building | ||
| 743 : | causing electrical noise and other problems. | ||
| 744 : | |||
| 745 : | \item Loops in the Earth system can cause interference. (The sound of ``mains hum'' one | ||
| 746 : | hears when turning sound equipment up to high volume is associated with these | ||
| 747 : | currents. In bad cases, this can prevent signals from being sent along cables.) | ||
| 748 : | |||
| 749 : | \item Earth is often connected to copper pipes that go into the ground. | ||
| 750 : | A faulty Earth connection can make a plumbing pipe live with a high | ||
| 751 : | voltage, causing a hazard. In one case, known to the author, a | ||
| 752 : | computer network cable had a voltage of 240V relative to a nearby chair. | ||
| 753 : | If the user had touched both at the same time, it would have led to a shock. | ||
| 754 : | |||
| 755 : | |||
| 756 : | \end{enumerate} | ||
| 757 : | Buildings and data centres must have a verfied Earth connection. They should also | ||
| 758 : | have fuses and circuit breakers to prevent large currents from flowing in the event | ||
| 759 : | of a fault. | ||
| 760 : | |||
| 761 : | |||
| 762 : | \chapter{Cooling and Airflow} | ||
| 763 : | |||
| 764 : | Electronic switching technology works by dumping current to ground | ||
| 765 : | through semi-conducting materials. This is the basic transistor modus | ||
| 766 : | operandi. This generates large amounts of heat. Essentially all of the | ||
| 767 : | electrical power consumed by a device ultimately becomes heat. | ||
| 768 : | |||
| 769 : | The aim of cooling is to prevent the temperature of devices becoming too great. | ||
| 770 : | If the heat increases, the electical resistance of the devices increases and | ||
| 771 : | even more energy is wasted as heat. Eventually, the electrical properties of the | ||
| 772 : | materials will no longer be suitable for their original purpose and the | ||
| 773 : | device will stop working. It could even melt. | ||
| 774 : | |||
| 775 : | \begin{itemize} | ||
| 776 : | \item Semiconductor chips can melt if they become too hot. | ||
| 777 : | |||
| 778 : | \item Heat increases electrical resistance, which in turn increases heat production | ||
| 779 : | since heat power conversion goes like $\sim I^2 R$. | ||
| 780 : | |||
| 781 : | \item Heat is a form of energy. Heat is not dangerous to equipment, but temperature is. | ||
| 782 : | Temperature is related to energy density. The more concentrated heat is, the higher the | ||
| 783 : | temperature. Thus we want to spread heat out. | ||
| 784 : | |||
| 785 : | \item If we are constantly producing more heat, then some of it has to be taken away | ||
| 786 : | to maintain the same temperature. | ||
| 787 : | |||
| 788 : | \item Air, water and liquified gases can be used to cool computing equipment, depending | ||
| 789 : | on the power they consume. Since these fluids are being used to carry | ||
| 790 : | heat, the rate of flow of heat away from computing equipment is important. | ||
| 791 : | |||
| 792 : | \end{itemize} | ||
| 793 : | |||
| 794 : | |||
| 795 : | \section{Heat and temperature} | ||
| 796 : | |||
| 797 : | Heat and temperature are related through a ``specific heat capacity''. All materials | ||
| 798 : | have a specific heat capacity which describes the approximate effect of heat on | ||
| 799 : | their temperature. | ||
| 800 : | |||
| 801 : | |||
| 802 : | The specific heat capacity (symbol $c_s$, also called specific heat) | ||
| 803 : | of a substance is defined as heat capacity per unit mass. The SI unit | ||
| 804 : | for specific heat capacity is the Joule per kilogram Kelvin, | ||
| 805 : | J·kg-1·K-1 or J/(kg·K), which is the amount of energy required to | ||
| 806 : | raise the temperature of one kilogram of the substance by one | ||
| 807 : | Kelvin. Heat capacity can be measured by using calorimetry. | ||
| 808 : | |||
| 809 : | Heat capacities are not really linear, but for small temperature variations we can think of | ||
| 810 : | heat capacity as being a constant property and we | ||
| 811 : | we can write down a linear formula for the relationship between temperature | ||
| 812 : | and heat energy. | ||
| 813 : | The formula is the same as the one we exploit for cooling: | ||
| 814 : | \beq | ||
| 815 : | \Delta Q = m c_s \Delta T | ||
| 816 : | \eeq | ||
| 817 : | where $\Delta Q$ is a change of heat energy. | ||
| 818 : | In other words temperature increase $\Delta T$, in a fixed mass $m$ of | ||
| 819 : | any substance, is proportional to the amount of heat added $\Delta$, | ||
| 820 : | assuming that the mass is constant. | ||
| 821 : | |||
| 822 : | |||
| 823 : | If we change the temperature by a lot, we might have to re-evaluate | ||
| 824 : | the heat capacity of the substance. Measuring the heat capacity of | ||
| 825 : | water produces different results if the starting point is 20 °C rather | ||
| 826 : | than 60 °C. Therefore the temperature at which the measurement was | ||
| 827 : | conducted must be specified for the value to be useful. | ||
| 828 : | |||
| 829 : | \subsection{Conduction and Convection} | ||
| 830 : | |||
| 831 : | \begin{itemize} | ||
| 832 : | \item {\em Conduction} is the direct transfer of heat by contact. For this, we use the | ||
| 833 : | heat capacity formula above. | ||
| 834 : | |||
| 835 : | \item {\em Convection} is a a transport of gas around a room due to the fact that, | ||
| 836 : | as a gas is heated its density decreases under constant pressure, so it gets | ||
| 837 : | lighter and rising. If the hot air rises, cooler air must fall underneath it. | ||
| 838 : | This results in a circulation of air called convection. | ||
| 839 : | \end{itemize} | ||
| 840 : | |||
| 841 : | How do we cool using these mechanisms? | ||
| 842 : | |||
| 843 : | If we bring a material into contact with a hotter substance, heat will | ||
| 844 : | flow into it by {\em conduction}. Then, if we can remove the hotter | ||
| 845 : | substance e.g. by flow convection, it will carry the heat away with it. | ||
| 846 : | Some values of $c_s$ for common | ||
| 847 : | cooling materials are: | ||
| 848 : | |||
| 849 : | \bigskip | ||
| 850 : | \begin{center} | ||
| 851 : | \begin{tabular}{l|l|l} | ||
| 852 : | Substance & Phase & Specific heat capacity\\ | ||
| 853 : | \hline | ||
| 854 : | Air (dry) & gas & 1005 J/kg/K\\ | ||
| 855 : | Air (100\% humidity) & gas & 1030 J/kg/K\\ | ||
| 856 : | Air (different source) & gas & 1158 J/kg/K\\ | ||
| 857 : | Water & liquid & 4186 J/kg/K\\ | ||
| 858 : | Copper (room temp) & solid & 385 J/kg/K\\ | ||
| 859 : | Gold (room temp) & solid & 129 J/kg/K\\ | ||
| 860 : | Silver (room temp) & solid & 235 J/kg/K\\ | ||
| 861 : | \end{tabular} | ||
| 862 : | \end{center} | ||
| 863 : | |||
| 864 : | \bigskip | ||
| 865 : | As we see, water is about four times as efficient at carrying heat per | ||
| 866 : | degree rise in temperature than is air. It is therefore | ||
| 867 : | correspondingly better at cooling than air. | ||
| 868 : | |||
| 869 : | |||
| 870 : | Convection also helps to keep heat-sources cool, since energy is literally | ||
| 871 : | carried away my the motion of the surrounding air or fluid. | ||
| 872 : | |||
| 873 : | \bigskip | ||
| 874 : | |||
| 875 : | \bigskip | ||
| 876 : | \framebox{ | ||
| 877 : | \begin{minipage}[t]{0.8\linewidth} | ||
| 878 : | |||
| 879 : | Note: Do not confuse the temperature of the air coming out of the | ||
| 880 : | computer with the temperature of the CPU. Think of when you open an | ||
| 881 : | oven door, after the inital rush of hot air. Metal objects in your | ||
| 882 : | oven can be hundreds of degrees Celcius, while the air inside is only | ||
| 883 : | 40-50 degrees Celcius. Bear in mind that cool air is flowing through a | ||
| 884 : | computer all the time with the aid of a fan. What you need to know is | ||
| 885 : | not the temperature of the air but the temperature of the components | ||
| 886 : | like the CPU and memory circuits. The plastics of these chips can melt | ||
| 887 : | if the computer gets too hot. | ||
| 888 : | |||
| 889 : | \end{minipage}} | ||
| 890 : | |||
| 891 : | \section{Melting temperatures} | ||
| 892 : | |||
| 893 : | It is not likely that heat from power dissipation will cause anything | ||
| 894 : | to melt. The first parts to melt would be the plastics and polymers: | ||
| 895 : | |||
| 896 : | \bigskip | ||
| 897 : | \begin{center} | ||
| 898 : | \begin{tabular}{l|l} | ||
| 899 : | Material & Melting Point\\ | ||
| 900 : | \hline | ||
| 901 : | Silicon & 1410 C\\ | ||
| 902 : | Copper & 1083 C\\ | ||
| 903 : | Gold & 1063 C\\ | ||
| 904 : | Silver & 879 C\\ | ||
| 905 : | Polymers& 50-100 C\\ | ||
| 906 : | \end{tabular} | ||
| 907 : | \end{center} | ||
| 908 : | |||
| 909 : | Today's motherboards come with temperature monitoring software and | ||
| 910 : | hardware which actually shuts the computer off the CPU temperature | ||
| 911 : | gets too hot. Even these however are not 100\% fail proof. | ||
| 912 : | |||
| 913 : | \bigskip | ||
| 914 : | \framebox{ | ||
| 915 : | \begin{minipage}[t]{0.8\linewidth} | ||
| 916 : | |||
| 917 : | CPU maximum operating temperatures lie between 60 and 90 degrees Celcius. | ||
| 918 : | |||
| 919 : | \end{minipage}} | ||
| 920 : | \bigskip | ||
| 921 : | |||
| 922 : | The reason for this is the way that semicondutors work. They rely on the | ||
| 923 : | ability to switch small voltages and currents on and off to simulate | ||
| 924 : | the behaviour of logic gates. To do this, it is necessary to have a | ||
| 925 : | semi-conductor gap, which electrons cannot cross without assistance. | ||
| 926 : | |||
| 927 : | As the temperature increases more and more electrons can bridge the | ||
| 928 : | gap without assistance and circuits cease to work, because everything is | ||
| 929 : | essentially always `switched on'. This happens when the size of the | ||
| 930 : | gap is of the same order as the thermal energy in the material. | ||
| 931 : | \beq | ||
| 932 : | eV \sim kT\\ | ||
| 933 : | 0.016 \times 10^{-19} \sim 10^{-23} \times 300K | ||
| 934 : | \eeq | ||
| 935 : | where $eV$ is the energy gap and $T$ is the absolute temperature, $k$ | ||
| 936 : | is Boltzmann's constant. With gap values of between 0.016-06 electron volts, | ||
| 937 : | semiconductors are in a fragile state above room temperature. | ||
| 938 : | |||
| 939 : | |||
| 940 : | \section{Strategy for heat transfer} | ||
| 941 : | |||
| 942 : | How do we keep things cool? | ||
| 943 : | |||
| 944 : | \subsection{Principles} | ||
| 945 : | \begin{itemize} | ||
| 946 : | \item {\em Convection} dominates heat flow in gases and liquids. | ||
| 947 : | If one prevents convection | ||
| 948 : | by placing obstacles that inhibit airflow, temperature will rise rapidly. | ||
| 949 : | |||
| 950 : | \item {\em Conduction} - transfer by contact (friction) | ||
| 951 : | Good conductors of electricity are generally also good conductors of heat, | ||
| 952 : | and vice versa. Thus we use metals as heat-sinks and radiators to carry away excess heat. | ||
| 953 : | |||
| 954 : | {\em Heat sinks} and | ||
| 955 : | {\em heat pipes} can be used to lead heat away from a hot-spot using | ||
| 956 : | a good conductor to spread heat over a wider area. Heat sinks are placed | ||
| 957 : | on the CPU in computers, for instance. | ||
| 958 : | |||
| 959 : | |||
| 960 : | \item {\em Radiation} - electromagnetic energy, or direct heat radiation. Heat will escape | ||
| 961 : | by photon radiation if it is not prevented from doing so by reflective walls. | ||
| 962 : | |||
| 963 : | Note that electromagnetic radiation only passed through transparent | ||
| 964 : | substances. However, a substance that is transparent to visible light | ||
| 965 : | is not necessarily transparent to heat, which has a much longer | ||
| 966 : | wavelength. This is the essence of the Greenhouse effect. Visible | ||
| 967 : | light can enter a Greenhouse through the glass walls, this energy is | ||
| 968 : | absorbed and some of the energy is radiated back as heat (some of it makes | ||
| 969 : | plants grow). However not all of the long wavelength heat energy passes | ||
| 970 : | through the glass. Some is absorbed by the air and heats it up. Eventually, | ||
| 971 : | it will cool off by radiation of very long wavelengths, but this process | ||
| 972 : | is much slower than the rate at which new energy is coming in, hence the | ||
| 973 : | energy density increases and temperature goes up. | ||
| 974 : | |||
| 975 : | \end{itemize} | ||
| 976 : | |||
| 977 : | \subsection{Heat extraction} | ||
| 978 : | |||
| 979 : | Cooling fans exploit the same idea as convection, to transport energy away | ||
| 980 : | by the actual movement of air (or sometimes fluid). | ||
| 981 : | \begin{figure}[ht] | ||
| 982 : | \epsfig{file=airflow.eps,width=12cm} | ||
| 983 : | \caption{Schematic illustration of air flowing through and around a rack device.} | ||
| 984 : | \end{figure} | ||
| 985 : | The rate of flow of volume is proportional to the fixed area of the channel | ||
| 986 : | cross-section and the length per unit time (velocity) of the coolant passing through it. | ||
| 987 : | \beq | ||
| 988 : | \frac{dV}{dt} &\propto& A \frac{dx}{dt}\\ | ||
| 989 : | F &\propto& A v | ||
| 990 : | \eeq | ||
| 991 : | You should bear in mind that | ||
| 992 : | \begin{itemize} | ||
| 993 : | \item It is the temperature of components inside the computer that is the main worry | ||
| 994 : | in cooling design. | ||
| 995 : | \item The temperature of the air coming out depends on many factors, including | ||
| 996 : | the ambient temperature around the box, the rate of air flow through | ||
| 997 : | the device and around it, the amount of convection in and around the | ||
| 998 : | device. humidity etc. | ||
| 999 : | |||
| 1000 : | \item Air is a poor carrier of heat, so if air is getting hot, you know that the | ||
| 1001 : | metalltic components are sizzling! | ||
| 1002 : | \end{itemize} | ||
| 1003 : | In very high speed computers that generate large amounts of heat quickly, | ||
| 1004 : | liquids such as water and nitrogen are use for cooling. The density of a liquid means | ||
| 1005 : | that it is able to carry much more heat, and hence a smaller flow is needed to carry a | ||
| 1006 : | larger amount of waste power. | ||
| 1007 : | |||
| 1008 : | \part{Equipment} | ||
| 1009 : | |||
| 1010 : | \chapter{Installing Blade Servers} | ||
| 1011 : | |||
| 1012 : | The blade servers in the lab contain pluggable micro-computers and an inbuilt | ||
| 1013 : | switch module that connects the array of blades to the outside world. | ||
| 1014 : | |||
| 1015 : | \section{Switch installation} | ||
| 1016 : | |||
| 1017 : | \begin{figure}[ht] | ||
| 1018 : | \epsfig{file=images/bay2.eps,width=10cm} | ||
| 1019 : | \caption{The empty I/O bay (bay 2) on the rear of the blade chassis.\label{bay2.eps}} | ||
| 1020 : | \end{figure} | ||
| 1021 : | \begin{figure}[ht] | ||
| 1022 : | \epsfig{file=images/bay2sw.eps,width=10cm} | ||
| 1023 : | \caption{The switch module is plugged into bay 2.\label{bay2sw.eps}} | ||
| 1024 : | \end{figure} | ||
| 1025 : | \begin{figure}[ht] | ||
| 1026 : | \epsfig{file=images/bay2swin.eps,width=10cm} | ||
| 1027 : | \caption{Switch module locks into place.\label{bay2swin.eps}} | ||
| 1028 : | \end{figure} | ||
| 1029 : | \begin{figure}[ht] | ||
| 1030 : | \epsfig{file=images/bay2swdone.eps,width=10cm} | ||
| 1031 : | \caption{The installed switch module.\label{bay2swdone.eps}} | ||
| 1032 : | \end{figure} | ||
| 1033 : | \begin{figure}[ht] | ||
| 1034 : | \epsfig{file=images/blade1.eps,width=10cm} | ||
| 1035 : | \caption{A blade server, unpacked.\label{blade1.eps}} | ||
| 1036 : | \end{figure} | ||
| 1037 : | \begin{figure}[ht] | ||
| 1038 : | \epsfig{file=images/openblade.eps,width=10cm} | ||
| 1039 : | \caption{Quick release pressure points on the side allow the cover to be removed.\label{openblade.eps}} | ||
| 1040 : | \end{figure} | ||
| 1041 : | \begin{figure}[ht] | ||
| 1042 : | \epsfig{file=images/insertdisk.eps,width=10cm} | ||
| 1043 : | \caption{A disk can be plugged in easily.\label{insertdisk.eps}} | ||
| 1044 : | \end{figure} | ||
| 1045 : | \begin{figure}[ht] | ||
| 1046 : | \epsfig{file=images/emptybay.eps,width=10cm} | ||
| 1047 : | \caption{A filler blade is removed to allow insertion of the server into the blade chassis.\label{emptybay.eps}} | ||
| 1048 : | \end{figure} | ||
| 1049 : | \begin{figure}[ht] | ||
| 1050 : | \epsfig{file=images/insert.eps,width=10cm} | ||
| 1051 : | \caption{The blade server slide in, with locking clamps extended.\label{insert.eps}} | ||
| 1052 : | \end{figure} | ||
| 1053 : | \begin{figure}[ht] | ||
| 1054 : | \epsfig{file=images/inplace.eps,width=10cm,angle=90} | ||
| 1055 : | \caption{A single blade server locked in place.\label{inplace.eps}} | ||
| 1056 : | \end{figure} | ||
| 1057 : | \begin{figure}[ht] | ||
| 1058 : | \epsfig{file=images/fullbay.eps,width=10cm} | ||
| 1059 : | \caption{Six blades installed. Note that the filler blades should be in place in the empty bays, in order to regulate the airflow through the cavity. If the fillers are not inserted, too much air is sucked through the opening, missing the actual blade servers.\label{fullbay.eps}} | ||
| 1060 : | \end{figure} | ||
| 1061 : | \begin{figure}[ht] | ||
| 1062 : | \epsfig{file=images/powerbutton.eps,width=10cm} | ||
| 1063 : | \caption{Blades are activated by a power button, underneath the cover. A keyboard, screen and CDROM switch also allow selection of the common managment resources.\label{powerbutton.eps}} | ||
| 1064 : | \end{figure} | ||
| 1065 : | \begin{figure}[ht] | ||
| 1066 : | \epsfig{file=images/poweron.eps,width=10cm} | ||
| 1067 : | \caption{Blades powered up.\label{poweron.eps}} | ||
| 1068 : | \end{figure} | ||
| 1069 : | \begin{figure}[ht] | ||
| 1070 : | \epsfig{file=images/bios.eps,width=10cm} | ||
| 1071 : | \caption{BIOS boot screen from the powered blade.\label{bios.eps}} | ||
| 1072 : | \end{figure} | ||
| 1073 : | |||
| 1074 : | |||
| 1075 : | |||
| 1076 : | \section{Switch configuration} | ||
| 1077 : | |||
| 1078 : | The management module of the blade chassis has a network interface as | ||
| 1079 : | well as a screen, keyboard, mouse connector. This ethernet connection | ||
| 1080 : | is on a private VLAN exclusively for connecting to the chassis blade | ||
| 1081 : | centre. | ||
| 1082 : | |||
| 1083 : | If this ethernet interface is connected to a switch with a DHCP service, | ||
| 1084 : | within two minutes of powering up the chassis, it will use DHCP to assign a | ||
| 1085 : | network address. If the two minutes expires, the default address | ||
| 1086 : | 192.168.70.125 is set. | ||
| 1087 : | |||
| 1088 : | We can therefore connect to it using a remote terminal, e.g. | ||
| 1089 : | |||
| 1090 : | \begin{enumerate} | ||
| 1091 : | \item Start a Linux laptop. | ||
| 1092 : | |||
| 1093 : | \item Connect to the laptop to the management interface using a crossover (null-modem) cable. | ||
| 1094 : | \item Configure the network interface of the terminal to be part of this | ||
| 1095 : | private subnet. e.g. | ||
| 1096 : | \begin{verbatim} | ||
| 1097 : | ifconfig eth0 192.168.70.2 netmask 255.255.255.0 broadcast 192.168.70.255 | ||
| 1098 : | \end{verbatim} | ||
| 1099 : | \item With this configuration, one can point a web browser to the addess | ||
| 1100 : | \begin{verbatim} | ||
| 1101 : | http://192.168.70.125 | ||
| 1102 : | \end{verbatim} | ||
| 1103 : | \end{enumerate} | ||
| 1104 : | |||
| 1105 : | We should enable INT1 - INT 6 for connect up the blades | ||
| 1106 : | internally. Then we should configure EXT1 to allow external | ||
| 1107 : | connection. For this we must enable trunking on VLAN1, which is the | ||
| 1108 : | default VLAN for the blades. | ||
| 1109 : | |||
| 1110 : | \begin{figure}[ht] | ||
| 1111 : | \epsfig{file=images/ibm-1.eps,width=15cm} | ||
| 1112 : | \caption{The web interface on 192.168.70.125 of the management console. Select I/O modules, Task management, Bay 2 and the Advanced Management option.} | ||
| 1113 : | \end{figure} | ||
| 1114 : | |||
| 1115 : | |||
| 1116 : | \begin{figure}[ht] | ||
| 1117 : | \epsfig{file=images/ibm-sw1.eps,width=16cm} | ||
| 1118 : | \caption{Entering the switch module management interface. Select, for instance, | ||
| 1119 : | the EXT1 interface, i.e. interface number 1 on the switch module we | ||
| 1120 : | installed in Bay 2. Note the dashboard and configure switches at the | ||
| 1121 : | top of the screen. Some options are only allowed in configure.} | ||
| 1122 : | \end{figure} | ||
| 1123 : | |||
| 1124 : | \begin{figure}[ht] | ||
| 1125 : | \epsfig{file=images/ibm-sw2.eps,width=16cm} | ||
| 1126 : | \caption{Inside the Advanced Management of the swicth, enable an external port for connecting the blade server chassis to an external switch or router. Enable external port and management of external port.} | ||
| 1127 : | \end{figure} | ||
| 1128 : | |||
| 1129 : | Once the external interface has been configured, the single external network | ||
| 1130 : | connection can be used to transfer the internal IP interfaces of the blade | ||
| 1131 : | servers to the outside world. One does not need to use a separate external | ||
| 1132 : | ethernet connecter per blade. The purpose of multiple interfaces is to allow | ||
| 1133 : | multiple VLANs. . | ||
| 1134 : | |||
| 1135 : | |||
| 1136 : | \section{Installation of Linux} | ||
| 1137 : | |||
| 1138 : | A number of CDs arrive with the blades. These contain drivers for MS Windows | ||
| 1139 : | and management software for RAID configuration. We shall not consider RAID | ||
| 1140 : | here, since our blades each have only a single disk. | ||
| 1141 : | |||
| 1142 : | None of the disks are needed for installation of Linux on the blade servers. | ||
| 1143 : | Any recent Linux will probably work. We have tested: | ||
| 1144 : | |||
| 1145 : | \begin{enumerate} | ||
| 1146 : | \item SuSE 10.1 | ||
| 1147 : | \end{enumerate} | ||
| 1148 : | |||
| 1149 : | Eahc blade appears to have two network interfaces which are not easily | ||
| 1150 : | distinguishable. Although both can be configured with an IP address | ||
| 1151 : | (even automatically with DHCP), Only one of the two works to connect | ||
| 1152 : | the blade server to the outside world. You will have to find out which | ||
| 1153 : | by trial and error. | ||
| 1154 : | |||
| 1155 : | If the switch communication has been enabled, | ||
| 1156 : | the interfaces are configured in the usual way for the operating system | ||
| 1157 : | concerned. | ||
| 1158 : | |||
| 1159 : | |||
| 1160 : | \part{Planning for High Volume Services} | ||
| 1161 : | |||
| 1162 : | |||
| 1163 : | \chapter{Designing an application architecture} | ||
| 1164 : | |||
| 1165 : | |||
| 1166 : | The architypical deisgn for network application services | ||
| 1167 : | is the so-called three tier architectures: webserver, application, database. | ||
| 1168 : | |||
| 1169 : | |||
| 1170 : | |||
| 1171 : | \section{Failure modes and redunancy} | ||
| 1172 : | |||
| 1173 : | Look up the Folk theorems about redundancy in the Analytical textbook. | ||
| 1174 : | Redundancy is key issue in handling quality of service. There is a | ||
| 1175 : | host of reasons why we might not be able to deliver on our promised | ||
| 1176 : | level of services. | ||
| 1177 : | \begin{figure}[ht] | ||
| 1178 : | \epsfig{file=serverrom.eps,width=6cm} | ||
| 1179 : | \caption{Load balaning is a tricky matter if one is looking for redundancy. | ||
| 1180 : | A load balancer is a tree -- which is a structure with many intrinsic points | ||
| 1181 : | of failure and bottlenecks.} | ||
| 1182 : | \end{figure} | ||
| 1183 : | |||
| 1184 : | When planning redudancy, there are certain thumb rules and theorems | ||
| 1185 : | concerning system reliability. Fig. \ref{folkth} illustrates the difference | ||
| 1186 : | between ``correct'' use of redundancy and a flawed strategy. | ||
| 1187 : | |||
| 1188 : | \begin{figure}[ht] | ||
| 1189 : | \epsfig{file=folkth.eps,width=6cm} | ||
| 1190 : | \caption{Redundancy folk theorem.\label{folkth}} | ||
| 1191 : | \end{figure} | ||
| 1192 : | |||
| 1193 : | |||
| 1194 : | Here is a minimum dpendency and redundancy strategy. Each of the doubled | ||
| 1195 : | components can be scaled up to $n$ to increase thoughput. | ||
| 1196 : | |||
| 1197 : | \begin{figure}[ht] | ||
| 1198 : | \epsfig{file=room.eps,width=6cm} | ||
| 1199 : | \caption{Minimum redundancy strategy\label{room}} | ||
| 1200 : | \end{figure} | ||
| 1201 : | |||
| 1202 : | \section{Basic queueing theory for capacity planning} | ||
| 1203 : | |||
| 1204 : | \begin{todo} | ||
| 1205 : | Read the papers by Bjørnstad and Burgess, and Burgess and Undheim about | ||
| 1206 : | how well queueing models reflect actual practice. | ||
| 1207 : | \end{todo} | ||
| 1208 : | |||
| 1209 : | Service provision is usually modelled as a relationship between a | ||
| 1210 : | client and a server (or service provider). A stream of requests | ||
| 1211 : | arrives randomly at the input of a system, at a rate of $\lambda$ | ||
| 1212 : | transactions per second, and they are added to a queue, whereupon they | ||
| 1213 : | are serviced at a rate $\mu$ transactions per second by a processor. | ||
| 1214 : | \index{Arrivals}\index{Queue} | ||
| 1215 : | |||
| 1216 : | Generalizations of this model include the possibility of multiple | ||
| 1217 : | (parallel) queues and multiple servers. In each case one considers a | ||
| 1218 : | single incoming stream of transaction requests; one then studies how to | ||
| 1219 : | cope with this stream of work. | ||
| 1220 : | |||
| 1221 : | As transaction demand is a random process, | ||
| 1222 : | queues are classified according to the type of arrival process for | ||
| 1223 : | requests, the type of completion process and the number of servers. In the | ||
| 1224 : | simplest form, Kendall notation of the form $A/S/n$ is used to classify | ||
| 1225 : | different queueing models. | ||
| 1226 : | \begin{itemize} | ||
| 1227 : | \item $A$: the arrival process distribution, e.g. Poisson arrival times, deterministic arrivals, or general spectrum arrivals. | ||
| 1228 : | \item $S$: the service completion distribution for job processing, e.g. Poisson renewals, etc. | ||
| 1229 : | \item $n$: the number of servers processing the incoming queue. | ||
| 1230 : | \end{itemize} | ||
| 1231 : | \index{Distribution!arrivals} | ||
| 1232 : | |||
| 1233 : | \subsection{The Poisson assumption} | ||
| 1234 : | |||
| 1235 : | \index{Distribution!Poisson} | ||
| 1236 : | \index{Distribution!Memoryless} | ||
| 1237 : | The distribution of inter-arrival times for both $A$ and $S$ is | ||
| 1238 : | normally considered to be a Poisson distribution in discrete time. One | ||
| 1239 : | writes this $M/M/n$ . This assumption is made in order to simplify the | ||
| 1240 : | analysis. $M$ stands for `memoryless' because the Poisson | ||
| 1241 : | distribution, taken over non-overlapping time intervals is a Markov | ||
| 1242 : | process whose behaviour, at any discrete time $t$, is quite independent of | ||
| 1243 : | what has happened in the past, i.e. it has no memory of the past. | ||
| 1244 : | The provides a huge simplification of the analysis. | ||
| 1245 : | |||
| 1246 : | Another reason for the Poisson assumption is that, in the limit of | ||
| 1247 : | large numbers of independent arrivals, one would expect the limiting | ||
| 1248 : | distribution to have a Poisson form. The argument, on very general | ||
| 1249 : | grounds is based on the fact that a Poisson distribution is the | ||
| 1250 : | limiting form of any Bernoulli process in which an event either occurs | ||
| 1251 : | or does not occur at time $t$. Suppose the probability of obtaining | ||
| 1252 : | a result is fixed and is equal to $p$ on each independent observation, then | ||
| 1253 : | the probability of obtaining $q$ positive arrivals in $n$ observations | ||
| 1254 : | is | ||
| 1255 : | \beq | ||
| 1256 : | P(q) = ^nC_q \;p^q (1-p)^{n-q}, | ||
| 1257 : | \eeq | ||
| 1258 : | where $^nC_r = n!/(n-r)!r!$ are the binomial coefficients. This is a | ||
| 1259 : | binomial distribution whose mean value is $\overline q = np$. Now, | ||
| 1260 : | suppose the probability of observing an arrival is scarce | ||
| 1261 : | i.e. $p\rightarrow 0$, but we consider the limit of long times or many | ||
| 1262 : | observations $n\rightarrow\infty$, taking the limit in such a way as to | ||
| 1263 : | make $np\rightarrow \lambda$, where $\lambda$ is a constant. Then, noting the | ||
| 1264 : | limits, | ||
| 1265 : | \beq | ||
| 1266 : | ^nC_r &\le& \frac{n^r}{r!}\nonumber\\ | ||
| 1267 : | ^nC_r &\ge& \frac{n^r}{r^r}\nonumber\\ | ||
| 1268 : | \lim_{\alpha\rightarrow\infty} (1-x)^\alpha &\rightarrow& {\rm e}^{-\alpha} ~~~{\rm where}~ (x < 1). | ||
| 1269 : | \eeq | ||
| 1270 : | one observes that | ||
| 1271 : | \beq | ||
| 1272 : | P(q=k) \le \frac{n^k}{k!}p^k(1-p)^{n-k} \rightarrow \frac{(np)^k}{k!}(1-p)^{n-k} | ||
| 1273 : | \rightarrow \frac{\lambda^k}{k!}{\rm e}^{-\lambda}, | ||
| 1274 : | \eeq | ||
| 1275 : | which is the the Poisson distribution, for which one verifies that | ||
| 1276 : | $\sum_{k=0}^\infty P(q=k) = 1$. | ||
| 1277 : | |||
| 1278 : | This widely held belief is somewhat controversial, however, as | ||
| 1279 : | measurements of network traffic have shown evidence of considerable | ||
| 1280 : | `burstiness', or long-tailed behaviour\cite{paxson2,paxson3,paxson4,leland2}. | ||
| 1281 : | Other work indicates that these contradictory measurements would in | ||
| 1282 : | fact settle into a Poisson distribution if only enough measurements | ||
| 1283 : | were taken\cite{cao2,hopcroft1}. However, it is estimated that | ||
| 1284 : | something of the order of $10^{10}$ transactions might be needed to | ||
| 1285 : | see this limiting form emerge. | ||
| 1286 : | |||
| 1287 : | Regardless of this controversy, the Poisson model survives in queueing | ||
| 1288 : | theory for its overriding simplicity. | ||
| 1289 : | \index{M/M/s queue} | ||
| 1290 : | |||
| 1291 : | \subsection{$M/M/s$ queues} | ||
| 1292 : | |||
| 1293 : | The basic ideas about queueing can be surmised from the simplest model | ||
| 1294 : | of a single server, memoryless queue: $M/M/1$. Then, let $n$ be the number | ||
| 1295 : | of unprocessed requests in the queue at time $t$, and suppose that | ||
| 1296 : | requests for transactions arrived at a rate of $\lambda$ per second, | ||
| 1297 : | and can be processed at a rate of $\mu$ per second. | ||
| 1298 : | |||
| 1299 : | We can treat this simple case as a continuum flow approximation, using | ||
| 1300 : | expectation values. | ||
| 1301 : | |||
| 1302 : | Consider any time $t$: the system makes a transition from a state of | ||
| 1303 : | queue length $n-1$ to $n$ at a continuous rate $\lambda$. Similarly, | ||
| 1304 : | when it is in state $n$, it makes transitions to a state of length $n$ | ||
| 1305 : | at a rate $\mu$. | ||
| 1306 : | The expectation of $\lambda$ arrivals per | ||
| 1307 : | second, when the queue length is in a state $n-1$ (for any $n$) must therefore | ||
| 1308 : | lead to the expectation of having $\mu$ completions when the state is | ||
| 1309 : | $n$, in order to balance the queue, so we write, on balance | ||
| 1310 : | $\lambda p_{n-1} = \mu p_n$ | ||
| 1311 : | or | ||
| 1312 : | \beq | ||
| 1313 : | p_n = \rho p_{n-1}, | ||
| 1314 : | \eeq | ||
| 1315 : | where $\rho = \lambda/\mu < 1$ is called the traffic intensity\cite{burgessbook2,jain1}. | ||
| 1316 : | This is a recurrence relation that can be solved for the entire distribution | ||
| 1317 : | $p_n$, for all $n$. One finds that | ||
| 1318 : | \beq | ||
| 1319 : | p_n = (1-\rho)\rho^n, | ||
| 1320 : | \eeq | ||
| 1321 : | and hence, the expected length of the queue is | ||
| 1322 : | \beq | ||
| 1323 : | \langle n\rangle = \sum_{n=0}^\infty p_n n = \frac{\rho}{1-\rho}. | ||
| 1324 : | \eeq | ||
| 1325 : | Clearly as the traffic intensity $\rho$ approaches unity, or $\lambda \rightarrow\mu$, | ||
| 1326 : | the queue length becomes infinite, as the server loses the ability to cope. | ||
| 1327 : | \index{Traffic intensity}\index{Intensity!traffic} | ||
| 1328 : | |||
| 1329 : | This situation improves somewhat for $s$ servers ($M/M/s$), where one finds | ||
| 1330 : | a much more complicated expression. In simplified form one has | ||
| 1331 : | \beq | ||
| 1332 : | \langle n \rangle = s\rho + P(n\ge s)\frac{\rho}{1-\rho} | ||
| 1333 : | \eeq | ||
| 1334 : | where the probability that the queue length exceeds the number | ||
| 1335 : | of servers $P(n\ge s)$ is of order $\rho^2$ for small load $\rho$, | ||
| 1336 : | which naturally leads to smaller queues. | ||
| 1337 : | |||
| 1338 : | It is possible to show that a single queue with $s$ servers is at least | ||
| 1339 : | as efficient as $s$ separate queues with their own server. This satisfies the | ||
| 1340 : | intuition that a single queue can be kept moving by any spare capacity | ||
| 1341 : | in any of its $s$ servers, whereas an empty queue that is separated from the | ||
| 1342 : | rest will simply be wasted capacity, while the others struggle with the load. | ||
| 1343 : | |||
| 1344 : | |||
| 1345 : | \subsection{Flow laws in the continuum approximation} | ||
| 1346 : | |||
| 1347 : | \index{Workflow}\index{Throughput} | ||
| 1348 : | Dimensional analysis provides us with some simple continuum relationships | ||
| 1349 : | for the `workflows'. | ||
| 1350 : | Consider the following definitions: | ||
| 1351 : | \beq | ||
| 1352 : | {\rm Arrival~rate~} \lambda &=& \frac{\rm No.~of~ arrivals}{\rm Time} = \frac{A}{T}\\ | ||
| 1353 : | {\rm Throughput~} \mu &=& \frac{\rm No.~of~completions}{\rm Time} = \frac{C}{T}\\ | ||
| 1354 : | {\rm Utilization~} U &=& \frac{\rm Busy~time}{\rm Total~time} = \frac{B}{T}\\ | ||
| 1355 : | {\rm Mean~service~time~} S &=& \frac{\rm Busy~Time}{\rm No.~of~completions} = \frac{B}{C}. | ||
| 1356 : | \eeq | ||
| 1357 : | \index{Utilization law} | ||
| 1358 : | The utilization $U$ tells us the mean level at which resources | ||
| 1359 : | are being scheduled in the system. The `utilization law' notes simply that: | ||
| 1360 : | \beq | ||
| 1361 : | U = \frac{B}{T} = \frac{C}{T}\times\frac{B}{C} | ||
| 1362 : | \eeq | ||
| 1363 : | or | ||
| 1364 : | \beq | ||
| 1365 : | U = \mu S. | ||
| 1366 : | \eeq | ||
| 1367 : | So utilization is proportional to the rate at which jobs are | ||
| 1368 : | completed and the mean time to complete a job. Thus it can be interpreted as | ||
| 1369 : | the probability that there is at least one job in the system. | ||
| 1370 : | \index{Utilization} | ||
| 1371 : | |||
| 1372 : | \begin{example} | ||
| 1373 : | Suppose a web server receives hits at a mean rate of 1.25 hits per second, | ||
| 1374 : | and the server takes an average of 2 milliseconds to reply. The law | ||
| 1375 : | tells us that the utilization of the server is | ||
| 1376 : | \beq | ||
| 1377 : | U = 1.25 \times 0.002 = 0.0025 = 0.25\%. | ||
| 1378 : | \eeq | ||
| 1379 : | This indicates to us that the system could probably work at four | ||
| 1380 : | hundred times this rate before saturation occurs, since $400\times | ||
| 1381 : | 0.25 = 100\%$. This conclusion is highly simplistic, but that is the | ||
| 1382 : | price for having a simple formula! | ||
| 1383 : | \end{example} | ||
| 1384 : | |||
| 1385 : | \subsection{Little's Law} | ||
| 1386 : | |||
| 1387 : | \index{Little's law} | ||
| 1388 : | Another dimensional truism is known as Little's law of queue size. It | ||
| 1389 : | says that the mean number of jobs in a queue $\langle n\rangle$, is | ||
| 1390 : | equal to the product of the mean arrival rate $\lambda$ (jobs per | ||
| 1391 : | second) and the mean response time $R$ (seconds) incurred by the | ||
| 1392 : | queue: | ||
| 1393 : | \beq | ||
| 1394 : | \langle n\rangle = \lambda R. | ||
| 1395 : | \eeq | ||
| 1396 : | Note that this equation has the generic form $V=IR$, similar to Ohm's | ||
| 1397 : | law in electricity. This an analogy is a direct consequence of a | ||
| 1398 : | simple balanced flow model. | ||
| 1399 : | |||
| 1400 : | In the M/M/1 queue, it is useful to characterize the expected response | ||
| 1401 : | time of the service centre. In other words, what is the likely time a | ||
| 1402 : | client will have to wait in order to have a task completed? From | ||
| 1403 : | Little's law, we know that the average number of tasks in a queue is | ||
| 1404 : | the product of the average response time and the average arrival rate, | ||
| 1405 : | so | ||
| 1406 : | \beq | ||
| 1407 : | R = \frac{Q_n}{\lambda}&=&\frac{\langle n\rangle}{\lambda}\nonumber\\ | ||
| 1408 : | &=& \frac{1}{\mu(1-\rho)}\nonumber\\ | ||
| 1409 : | &=& \frac{1}{(\mu-\lambda)}. | ||
| 1410 : | \eeq | ||
| 1411 : | Notice that this is finite as long as $\lambda \ll \mu$, but as | ||
| 1412 : | $\lambda \rightarrow \mu$, the response time becomes unbounded. | ||
| 1413 : | |||
| 1414 : | Load balancing over queues is a topic for more advanced queueing | ||
| 1415 : | theory. Weighted fair queueing and algorithms like round-robin etc., | ||
| 1416 : | can be used to spread the load of an incoming queue amongst multiple | ||
| 1417 : | servers, to best exploit their availability. Readers are referred to | ||
| 1418 : | refs. \cite{burgessbook2,jain1,queues1,queues2} for more discussion on this. | ||
| 1419 : | \index{Load balancing} | ||
| 1420 : | |||
| 1421 : | |||
| 1422 : | \chapter{Capacity Planning} | ||
| 1423 : | |||
| 1424 : | Let's now see how we can use the queueing results to estimate our needs for | ||
| 1425 : | a system. Most service systems are I/O bound, i.e. the chief bottleneck is | ||
| 1426 : | input/output. However, in a data centre environment we have another problem | ||
| 1427 : | that we do not experience on a small scale: power limitations. | ||
| 1428 : | |||
| 1429 : | Before designing a data center we need to work out some basic numbers, the | ||
| 1430 : | scales and sizes of numbers involved. | ||
| 1431 : | \begin{itemize} | ||
| 1432 : | \item How fast can we process jobs? (CPU rate and number of parallel servers). | ||
| 1433 : | |||
| 1434 : | \item How much memory do we need: how many jobs will be in the system (queue length) | ||
| 1435 : | and what is the average size of a job in memory? | ||
| 1436 : | |||
| 1437 : | \item How many bytes per second of network arrivals and returns do we expect? | ||
| 1438 : | |||
| 1439 : | \item How many bytes per second of disk activity do we expect? | ||
| 1440 : | |||
| 1441 : | \item What is the power cost (including cooling) of running out servers? | ||
| 1442 : | |||
| 1443 : | \end{itemize} | ||
| 1444 : | |||
| 1445 : | \begin{todo} | ||
| 1446 : | Collect some facts and figures about today's technologies. Don't just choose the | ||
| 1447 : | latest and greatest, but also some odler slower technologies that are still in use. | ||
| 1448 : | \begin{itemize} | ||
| 1449 : | \item CPU speeds in Hz (cycles per second). Find out for RISC and CISC separately, | ||
| 1450 : | and, in the case of CISC, find out the average number of clock cycles per instruction. | ||
| 1451 : | Convert both of these into instructions per second. | ||
| 1452 : | \item Disk transfer rates in MB/s (SCSI/SATA). | ||
| 1453 : | \item Network speeds. | ||
| 1454 : | \item How much does a PC cost? | ||
| 1455 : | \item How much do different kinds of disks cost? | ||
| 1456 : | \item How much does electrical power cost? | ||
| 1457 : | \end{itemize} | ||
| 1458 : | \end{todo} | ||
| 1459 : | |||
| 1460 : | \section{Simple queue estimates} | ||
| 1461 : | |||
| 1462 : | There is a viewpoint that says, in most systems there will be a point of bottleneck | ||
| 1463 : | somewhere, and so we should model most systems as M/M/1. Let's use the basic | ||
| 1464 : | M/M/1 formulae to make some calculations | ||
| 1465 : | |||
| 1466 : | \begin{example} | ||
| 1467 : | Use the formulae for the M/M/1 queue to calculate the amount of RAM, | ||
| 1468 : | disk rate and CPU speed to handle the average loads: | ||
| 1469 : | \begin{enumerate} | ||
| 1470 : | \item 10 downloads per second of image files 20MB each. | ||
| 1471 : | \item 100 downloads per second of image files 20MB each. | ||
| 1472 : | \item 20 downloads per second of resized images to a mobile phone, each image is 20MB and | ||
| 1473 : | takes 100 CPU cycles per byte render and convert in RAM. | ||
| 1474 : | \end{enumerate} | ||
| 1475 : | given that the size of the download request is of the order 100 bytes. | ||
| 1476 : | \end{example} | ||
| 1477 : | |||
| 1478 : | \begin{enumerate} | ||
| 1479 : | |||
| 1480 : | \item If we have 10 downloads per second, each of 20 MB/s, then we need to process | ||
| 1481 : | \beq | ||
| 1482 : | 10 \times 20 = 200 MB/s. | ||
| 1483 : | \eeq | ||
| 1484 : | Ideally, all of the components in the system will be able to handle | ||
| 1485 : | data at this rate or faster. (Note that when $\lambda = \mu$, the | ||
| 1486 : | queue length is not zero but infinite -- can you see why?) | ||
| 1487 : | Let us suppose that the system disk has an average performance of | ||
| 1488 : | 500 MB/s, then we have: | ||
| 1489 : | \beq | ||
| 1490 : | \lambda = \frac{200}{20} = 10 j/s\nonumber\\ | ||
| 1491 : | \mu = \frac{500}{20} = 25 j/s. | ||
| 1492 : | \eeq | ||
| 1493 : | Note that we cannot use MB/s for $\lambda$, or the formulae will be | ||
| 1494 : | wrong. In the worst case, each component of the system must be able | ||
| 1495 : | to handle this data rate. | ||
| 1496 : | |||
| 1497 : | The average queue length: | ||
| 1498 : | \beq | ||
| 1499 : | \langle n\rangle = \frac{\rho}{1-\rho} = \frac{\lambda}{\mu-\lambda} = 5/3 \simeq 3 | ||
| 1500 : | \eeq | ||
| 1501 : | This tells us that there will be able 3 jobs in the system queue waiting to be | ||
| 1502 : | processed at any given moment. That means the RAM we need to service the queue | ||
| 1503 : | will be $3\times 100$ bytes, since each job request was 100 bytes long. Clearly | ||
| 1504 : | RAM is not much of an issue for a lightly loaded server. | ||
| 1505 : | |||
| 1506 : | The mean response time for the service is | ||
| 1507 : | \beq | ||
| 1508 : | R = \frac{1}{\mu-\lambda} = \frac{1}{25-10} = \frac{1}{15} sec. | ||
| 1509 : | \eeq | ||
| 1510 : | |||
| 1511 : | \item With the numbers above the server would fail long before this limit, since | ||
| 1512 : | at queue size blows up to infinity as $\lambda\rightarrow 25$ and the server starts thrashing, i.e. it spends all its time managing the queue and not completing jobs.. | ||
| 1513 : | |||
| 1514 : | \item Now we have an added processing time. Suppose we assume that the CPU has a clock | ||
| 1515 : | speed of $10^9$ Hz, and an average of 4 cycles per instruction. Then we process | ||
| 1516 : | at a rate of 100 CPU cycles per byte, for all 20MB. | ||
| 1517 : | \beq | ||
| 1518 : | \frac{20 \times 10^6 \times 100}{10^9} = 2s. | ||
| 1519 : | \eeq | ||
| 1520 : | Thus we have a processing overhead of 2 seconds per download, which must be | ||
| 1521 : | added to the normal queue response time. | ||
| 1522 : | \beq | ||
| 1523 : | \lambda = \frac{400}{20} = 20 j/s\nonumber\\ | ||
| 1524 : | \mu = \frac{500}{20} = 25 j/s.\nonumber\\ | ||
| 1525 : | \langle n\rangle = \frac{\rho}{1-\rho} = \frac{\lambda}{\mu-\lambda} = 4\nonumber\\ | ||
| 1526 : | R = \frac{1}{\mu-\lambda} + CPU = \frac{1}{25-20} + CPU = \frac{1}{5} + 2 sec. | ||
| 1527 : | \eeq | ||
| 1528 : | Notice that the CPU processing now dominates the service time, so we should look | ||
| 1529 : | at increasing the speed of the CPU before worrying about anything else. | ||
| 1530 : | \end{enumerate} | ||
| 1531 : | |||
| 1532 : | |||
| 1533 : | \section{Peak load handling} | ||
| 1534 : | |||
| 1535 : | There is always a risk in service provision that (at some time) a | ||
| 1536 : | level of demand will occur that is more than we can serve. Clearly we | ||
| 1537 : | would like to avoid such a time. | ||
| 1538 : | |||
| 1539 : | One way to handle this is to design a system that can be extended on demand. | ||
| 1540 : | |||
| 1541 : | \section{Power and cooling} | ||
| 1542 : | |||
| 1543 : | It is quite common for inexperienced staff to underestimate the power | ||
| 1544 : | and cooling requirements for a data centre. Today, with blade chassis | ||
| 1545 : | computers, it is possible to pack even more computers into a small | ||
| 1546 : | space than ever before. | ||
| 1547 : | |||
| 1548 : | More computers means more power, and more cooling. Moreover, it is | ||
| 1549 : | important to be able to deliver peak power without tripping a fuse or | ||
| 1550 : | circuit-breaker. As we have learned in earlier in this booklet, the | ||
| 1551 : | current is the killer. | ||
| 1552 : | |||
| 1553 : | Some conservative numbers on a website, rating a blade chassis at 2kW | ||
| 1554 : | does not mean that we can get away with a 10A circuit. The blades | ||
| 1555 : | might peak at a significantly higher usage. Moreover, blade servers | ||
| 1556 : | are designed with redundant power supplies, meaning that everything | ||
| 1557 : | should be doubled. That could mean that we end up requiring two | ||
| 1558 : | independent 20A circuits {\em per blade chassis}. A typical rack can | ||
| 1559 : | house six or more chassix -- and that is almost 30kW (30VA) of power | ||
| 1560 : | multiplied by two for redundancy. | ||
| 1561 : | |||
| 1562 : | Note that we add to the power requirement for the computing racks the | ||
| 1563 : | power required to cool them. If you are using air conditioning -- that | ||
| 1564 : | will add a significant cost to your electricity bill. | ||
| 1565 : | |||
| 1566 : | We are talking about a power supply to rival any industrial production line. | ||
| 1567 : | |||
| 1568 : | \subsection{Connection} | ||
| 1569 : | |||
| 1570 : | Data centres should be wired by experienced electrical engineers. | ||
| 1571 : | Some designs require three phase power delivery, most today have both | ||
| 1572 : | options. Not all electricians are experienced in these matters. Try to | ||
| 1573 : | contact a company that had experience of high current installations. | ||
| 1574 : | |||
| 1575 : | \subsection{Power saving} | ||
| 1576 : | |||
| 1577 : | Because we use more power by running CPUs at high frequency, there is | ||
| 1578 : | a calculation to do to see if we can save some power | ||
| 1579 : | by running more computers at a lower frequency. | ||
| 1580 : | \begin{todo} | ||
| 1581 : | See the article about power management by Burgess | ||
| 1582 : | and Sandnes in this compendium. How relevant is this article to power | ||
| 1583 : | saving in the data centre? Explain your thoughts, critically. | ||
| 1584 : | \end{todo} | ||
| 1585 : | |||
| 1586 : | \begin{todo} | ||
| 1587 : | In the article, it is shown that power increases with the square of | ||
| 1588 : | the CPU frequency. Can you predict, based on this relationship, the | ||
| 1589 : | shape of the curve of average power consumptions versus number of | ||
| 1590 : | CPUs, at constant processing rate. Can you measure this curve experimentally? | ||
| 1591 : | \end{todo} | ||
| 1592 : | |||
| 1593 : | \begin{todo} | ||
| 1594 : | Study the example calculation in 7.6 of the article System | ||
| 1595 : | Administration and the Business Process. Use the price estimates for | ||
| 1596 : | electricity to draw up an estimate for how long it would take to pay for | ||
| 1597 : | an extra server, given the savings in power and cooling. | ||
| 1598 : | \end{todo} | ||
| 1599 : | |||
| 1600 : | |||
| 1601 : | \subsection{UPS - Uninterruptible Power Supply} | ||
| 1602 : | |||
| 1603 : | \subsection{Building design} | ||
| 1604 : | |||
| 1605 : | With so much heat being wasted in such a small area, we have to think | ||
| 1606 : | seriously in the future about how buildings are designed. How do we | ||
| 1607 : | re-use the power? This topic would make an interesting master project. | ||
| 1608 : | |||
| 1609 : | |||
| 1610 : | \chapter{Performance Measurement and Benchmarking} | ||
| 1611 : | |||
| 1612 : | Refer to the book by Jain for more details\cite{jain1}. | ||
| 1613 : | |||
| 1614 : | \section{Planning} | ||
| 1615 : | |||
| 1616 : | Steps in planning: | ||
| 1617 : | \begin{itemize} | ||
| 1618 : | \item Place measurement instrumentation. | ||
| 1619 : | \item Collect data. | ||
| 1620 : | \item Characterize and understand workload patterns. | ||
| 1621 : | \item Model and predict performance alternatives. | ||
| 1622 : | \end{itemize} | ||
| 1623 : | |||
| 1624 : | |||
| 1625 : | \section{Common mistakes} | ||
| 1626 : | |||
| 1627 : | \begin{itemize} | ||
| 1628 : | |||
| 1629 : | \item Workload only represents average behaviour -- no outlier stress points. | ||
| 1630 : | \item Not instrumenting enough to find the source of I/O bottlenecks. | ||
| 1631 : | \item Measurement uncertainties are ignored. | ||
| 1632 : | \item Ignoring measurement overhead. | ||
| 1633 : | \item Not repeating or validating measurements many times to eliminate random uncertainties. | ||
| 1634 : | \item Not ensuring same conditions. | ||
| 1635 : | \item Not measuring performance under transient behaviour (rapid bursts) | ||
| 1636 : | \item Collecting data but not analyzing properly. | ||
| 1637 : | \end{itemize} | ||
| 1638 : | |||
| 1639 : | %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% | ||
| 1640 : | |||
| 1641 : | \part{Load Balancing} | ||
| 1642 : | |||
| 1643 : | \chapter{DNS Load Balancing} | ||
| 1644 : | |||
| 1645 : | %\newtheorem{example}{Example} | ||
| 1646 : | %\newtheorem{question}{Question} | ||
| 1647 : | |||
| 1648 : | \section{Load Balancing} | ||
| 1649 : | What do you do when your web server is overloaded with heavy requests? | ||
| 1650 : | One suggestion is to add hardware upgrades to the server, such as a | ||
| 1651 : | stronger processor and more memory. Another idea is to support the web | ||
| 1652 : | site with several servers. Using some kind of request scheduling | ||
| 1653 : | mechanism, it is then possible to share the load among the available | ||
| 1654 : | servers. This example, though trivial, outlines the essence of load | ||
| 1655 : | balancing: Sharing an imposed workload between several ordinary and | ||
| 1656 : | relatively inexpensive servers, as opposed to handling the entire load | ||
| 1657 : | on a single high-end node. In simplified queueing theory terms, we are | ||
| 1658 : | moving from $M/M/1$ to $M/M/k$ or $M/M/1^k$ queues. In other words, we | ||
| 1659 : | substitute the single server setup with more servers or service | ||
| 1660 : | channels. | ||
| 1661 : | |||
| 1662 : | Sharing load between servers not only has direct economical effects, | ||
| 1663 : | it often brings about welcome side effects. One such benefit is the | ||
| 1664 : | level of redundancy implied by running a load balanced service. Say | ||
| 1665 : | for example that you serve a corporate web site between ten servers. | ||
| 1666 : | Then you might afford to lose one or more servers without severely | ||
| 1667 : | impacting the browsing experience for customers. This approach can | ||
| 1668 : | easily alleviate software or hardware faults in single nodes. It | ||
| 1669 : | should be noted, however, that a proper redundant system requires | ||
| 1670 : | redundancy in all links of the chain. | ||
| 1671 : | |||
| 1672 : | %On the issue of manageability, keeping track of software and general | ||
| 1673 : | %operation of a computer cluster can be considered a drawback compared | ||
| 1674 : | %to the monolithic approach. However, with the right tools and | ||
| 1675 : | %configuration, difficulty of management need not turn out to be the | ||
| 1676 : | %case at all. | ||
| 1677 : | |||
| 1678 : | It is apparent that the difference between lower and higher levels of | ||
| 1679 : | load balancing is directly related to granularity and scalability, | ||
| 1680 : | both when it comes to data handling and for manageability. For | ||
| 1681 : | example, in a multi-CPU system the task scheduler in the operating | ||
| 1682 : | system organises processes and threads so that they are load-balanced | ||
| 1683 : | on the available CPU resources. On this scale we deal with units of | ||
| 1684 : | CPU cycles and microsecond deltas, which are not directly relevant | ||
| 1685 : | when we talk about server or network load balancing. However, if we | ||
| 1686 : | zoom out several orders of magnitude and look at network topologies, | ||
| 1687 : | we can typically operate with units of packets and bytes, and typical | ||
| 1688 : | deltas of seconds to hours. This is where load balancing becomes more | ||
| 1689 : | interesting and manageable in system administration. Towards the | ||
| 1690 : | topmost levels are considerations such as geographic proximity, | ||
| 1691 : | routing mechanisms and service level agreements. These are interesting | ||
| 1692 : | because they move away from details and allow us to get a handle on | ||
| 1693 : | the big picture of large-scale network traffic and do load balancing | ||
| 1694 : | of that very same traffic. This implies that we deal with trends and | ||
| 1695 : | analysis of large amounts of data over very long time spans, typically | ||
| 1696 : | hours and days, to months and even years. | ||
| 1697 : | |||
| 1698 : | \section{DNS-based Load Balancing} | ||
| 1699 : | The Domain Name System (DNS) is the global name lookup service in the | ||
| 1700 : | Internet. It is a distributed, hierarchical and redundant database | ||
| 1701 : | running on thousands of servers worldwide, each responsible for one or | ||
| 1702 : | more DNS zones. When a client issues a request for a URL, the browser | ||
| 1703 : | will first try to resolve the hostname in the URL into an IP address, | ||
| 1704 : | so that it knows where to send the request. This is where it is | ||
| 1705 : | possible for a DNS server to influence the outcome of a query, | ||
| 1706 : | effectively directing the client to a desired or lightly loaded site. | ||
| 1707 : | |||
| 1708 : | While DNS in itself was not designed particularly with the | ||
| 1709 : | requirements of load balancing in mind, it does provide a limited set | ||
| 1710 : | of options to facilitate simple balancing schemes. However, together | ||
| 1711 : | with dynamical back-ends, DNS can be a well suited front-end to robust | ||
| 1712 : | load balancing mechanisms. The strength of using DNS is its almost | ||
| 1713 : | universal deployment: Practically every node connected to an IP | ||
| 1714 : | network is capable of DNS resolving. This makes DNS one of the most | ||
| 1715 : | desirable \emph{interfaces} for global load sharing implementations. | ||
| 1716 : | Dynamical back-ends are discussed later. | ||
| 1717 : | |||
| 1718 : | To give an insight into how nameservers respond to queries, consider | ||
| 1719 : | the listing below, showing what a client would receive (per July 2006) | ||
| 1720 : | when asking for the IP address of the cnn.com host. This particular | ||
| 1721 : | output is from the `dig' application, and shows three main sections: | ||
| 1722 : | The question section echoes the request information; the answer | ||
| 1723 : | section shows the matching records (if any); the authority section | ||
| 1724 : | lists the authoritative nameservers for the domain. | ||
| 1725 : | |||
| 1726 : | \setlength{\fboxsep}{0pt} | ||
| 1727 : | \begin{code} | ||
| 1728 : | \fvset{numbers=left,commandchars=\\\{\}} | ||
| 1729 : | \begin{Verbatim} | ||
| 1730 : | ;; QUESTION SECTION: | ||
| 1731 : | ;cnn.com. IN A | ||
| 1732 : | |||
| 1733 : | ;; ANSWER SECTION: | ||
| 1734 : | cnn.com. 300 IN A 64.236.29.120 ;This | ||
| 1735 : | cnn.com. 300 IN A 64.236.16.20 ;is | ||
| 1736 : | cnn.com. 300 IN A 64.236.16.52 ;an | ||
| 1737 : | cnn.com. 300 IN A 64.236.16.84 ;RRset | ||
| 1738 : | cnn.com. 300 IN A 64.236.16.116 ;with | ||
| 1739 : | cnn.com. 300 IN A 64.236.24.12 ;eight | ||
| 1740 : | cnn.com. 300 IN A 64.236.24.20 ;web | ||
| 1741 : | cnn.com. 300 IN A 64.236.24.28 ;servers | ||
| 1742 : | |||
| 1743 : | ;; AUTHORITY SECTION: | ||
| 1744 : | cnn.com. 600 IN NS twdns-04.ns.aol.com. | ||
| 1745 : | cnn.com. 600 IN NS twdns-01.ns.aol.com. | ||
| 1746 : | cnn.com. 600 IN NS twdns-02.ns.aol.com. | ||
| 1747 : | cnn.com. 600 IN NS twdns-03.ns.aol.com. | ||
| 1748 : | \end{Verbatim} | ||
| 1749 : | \end{code} | ||
| 1750 : | |||
| 1751 : | There are a few interesting properties of the DNS response in the | ||
| 1752 : | listing. First of all, it is clear that the nameservers return | ||
| 1753 : | multiple A-records for the cnn.com hostname -- this is known as a | ||
| 1754 : | resource record set (RRset). It is difficult to determine whether | ||
| 1755 : | there is a complex algorithm behind the result or if it is simply | ||
| 1756 : | a random permutation of the available servers. Because ordering of | ||
| 1757 : | RRsets is never guaranteed, it is likely that the DNS servers use | ||
| 1758 : | random scheduling. In other words, they do not map server preference | ||
| 1759 : | to the RRset ordering, such as placing the most preferred server on | ||
| 1760 : | top of the list. | ||
| 1761 : | |||
| 1762 : | Another point to notice is the time-to-live (TTL) values in the second | ||
| 1763 : | column indicate that the returned records have a very short caching | ||
| 1764 : | lifetime, 5 and 10 minutes for the A and NS records, respectively. A | ||
| 1765 : | low TTL ensures that DNS servers are queried often, and are thus | ||
| 1766 : | able to influence the answer over relatively small time intervals, | ||
| 1767 : | e.g.~removing dead servers from the RRset. Low TTLs come at the cost | ||
| 1768 : | of higher frequency of queries, which can add a considerable delay to | ||
| 1769 : | the total page response time. | ||
| 1770 : | |||
| 1771 : | \section{Configuring BIND} | ||
| 1772 : | A simple setup of DNS load balancing in the BIND software takes very | ||
| 1773 : | little effort. The following listing is an example taken from a normal | ||
| 1774 : | BIND zone file with standard syntax, and it describes the fictional | ||
| 1775 : | \url{test.lan} zone: | ||
| 1776 : | \setlength{\fboxsep}{0pt} | ||
| 1777 : | \begin{code} | ||
| 1778 : | \fvset{numbers=left,commandchars=\\\{\}} | ||
| 1779 : | \begin{Verbatim} | ||
| 1780 : | \$TTL 60 | ||
| 1781 : | test.lan. IN SOA ns.test.lan. hostmaster.test.lan. ( | ||
| 1782 : | 1 ; Serial | ||
| 1783 : | 604800 ; Refresh | ||
| 1784 : | 86400 ; Retry | ||
| 1785 : | 2419200 ; Expire | ||
| 1786 : | 604800 ) ; Negative Cache TTL | ||
| 1787 : | ; | ||
| 1788 : | test.lan. IN NS ns.test.lan. ; NameServer record. | ||
| 1789 : | ns IN A 10.0.0.1 ; A-record for nameserver. | ||
| 1790 : | www IN A 10.0.0.13 | ||
| 1791 : | www IN A 10.0.0.11 | ||
| 1792 : | www IN A 10.0.0.14 | ||
| 1793 : | www IN A 10.0.0.16 | ||
| 1794 : | www IN A 10.0.0.15 | ||
| 1795 : | www IN A 10.0.0.12 | ||
| 1796 : | \end{Verbatim} | ||
| 1797 : | \end{code} | ||
| 1798 : | The first line defines the time-to-live value to use for all the | ||
| 1799 : | records contained within the zonefile. Next, on lines 2 through 7, is | ||
| 1800 : | the \emph{start of authority} record, which describes behaviour of | ||
| 1801 : | slave nameservers. The relevant records are the six A-records that | ||
| 1802 : | make up the RRset for the \url{www.test.lan} hostname. As a result, | ||
| 1803 : | any lookup for the A-record for \url{www.test.lan} would return the | ||
| 1804 : | entire set of addresses to the querying client. The default scheduling | ||
| 1805 : | mechanism in BIND has been known to vary with releases, but is | ||
| 1806 : | typically set to either random or round-robin with an arbitrary | ||
| 1807 : | starting order. It can be configured in the \emph{named} configuration | ||
| 1808 : | options, using the directive ``rr-set ordering''. | ||
| 1809 : | |||
| 1810 : | \section{Dynamical DNS Zones} | ||
| 1811 : | The current (July 2006) DNS scheme deployed on \url{amazon.com} offers | ||
| 1812 : | another approach to load sharing. When querying the DNS servers for | ||
| 1813 : | the A record of \url{www.amazon.com}, only one address is returned. | ||
| 1814 : | Subsequent queries show that the server returns a single IP address | ||
| 1815 : | out of a small set. The effects of this approach are twofold. On one | ||
| 1816 : | side it lets the DNS servers decide which server the client connects | ||
| 1817 : | to -- i.e. the load sharing is very much more controlled than in the | ||
| 1818 : | previous CNN example. On the other hand this solution introduces a | ||
| 1819 : | single point of failure for short periods of time: The client is given | ||
| 1820 : | only one point of correspondence; should it fail, the client must give | ||
| 1821 : | up or wait for the TTL to expire. By then, Amazon might have lost | ||
| 1822 : | their customer. | ||
| 1823 : | |||
| 1824 : | Is there a middle ground here, that is, a compromise between load | ||
| 1825 : | sharing control and availability? Such a compromise could be | ||
| 1826 : | implemented by having the DNS servers reply with a \emph{subset} of | ||
| 1827 : | the full list of servers. It can be argued that this solution provides | ||
| 1828 : | both the control and availability needed for a high-volume site. | ||
| 1829 : | |||
| 1830 : | When implementing dynamical DNS zones, there is some choice of software. | ||
| 1831 : | For many years the PowerDNS package has been a viable alternative to | ||
| 1832 : | BIND. PowerDNS is an authoritative-only DNS server with a very | ||
| 1833 : | flexible back-end architecture: It supports loading zones from plain | ||
| 1834 : | BIND-style zone files, a variety of SQL databases, LDAP, etc. Examples | ||
| 1835 : | of use include setting server preference using a weight factor, or | ||
| 1836 : | limiting the result to a subset of the matching A-records. | ||
| 1837 : | |||
| 1838 : | On a side note, the latest version of BIND, v9.4, features the DLZ | ||
| 1839 : | extension. DLZ, dynamically loadable zones, gives BIND much of the | ||
| 1840 : | same back-end capabilities as PowerDNS when it comes to serving | ||
| 1841 : | dynamical content. | ||
| 1842 : | |||
| 1843 : | \section{Configuring PowerDNS} | ||
| 1844 : | Configuring PowerDNS zones is done by populating a database of choice | ||
| 1845 : | with the zone information, and then specifying the SQL query needed to | ||
| 1846 : | fetch the data in the configuration file. An example of a query from a | ||
| 1847 : | populated database is shown below. Note that it contains mostly the | ||
| 1848 : | same information as the static zone file (though the SOA content is | ||
| 1849 : | truncated): | ||
| 1850 : | \setlength{\fboxsep}{0pt} | ||
| 1851 : | \begin{code} | ||
| 1852 : | \fvset{numbers=left} | ||
| 1853 : | \begin{Verbatim} | ||
| 1854 : | # select id,name,type,content,ttl,pref from records; | ||
| 1855 : | |||
| 1856 : | id | name | type | content | ttl | pref | ||
| 1857 : | ----+--------------+------+---------------------+------+------ | ||
| 1858 : | 1 | test.lan | SOA | ns.test.lan hostmast| 86400| | ||
| 1859 : | 2 | test.lan | NS | ns.test.lan | 86400| | ||
| 1860 : | 9 | ns.test.lan | A | 10.0.0.1 | 86400| | ||
| 1861 : | 3 | www.test.lan | A | 10.0.0.11 | 30| 10 | ||
| 1862 : | 4 | www.test.lan | A | 10.0.0.12 | 30| 15 | ||
| 1863 : | 5 | www.test.lan | A | 10.0.0.13 | 30| 60 | ||
| 1864 : | 6 | www.test.lan | A | 10.0.0.14 | 30| 45 | ||
| 1865 : | 7 | www.test.lan | A | 10.0.0.15 | 30| 100 | ||
| 1866 : | 8 | www.test.lan | A | 10.0.0.16 | 30| 30 | ||
| 1867 : | \end{Verbatim} | ||
| 1868 : | \end{code} | ||
| 1869 : | By appending the ``ORDER BY random() LIMIT 1'' expression to the SQL | ||
| 1870 : | query, only one random record is returned (the Amazon case). | ||
| 1871 : | Similarly, ``ORDER BY pref LIMIT 3'' would return the three most | ||
| 1872 : | preferable servers. If the \emph{pref} value is static, the three | ||
| 1873 : | least preferable servers would never be used. This can be solved by | ||
| 1874 : | making the pref value dynamically updated using a simple health-check | ||
| 1875 : | system. An example of such a system would be monitoring server load in | ||
| 1876 : | short intervals, and updating the pref value to reflect the current | ||
| 1877 : | load scenario. | ||
| 1878 : | |||
| 1879 : | The practical uses of dynamical zones are many and vary beyond simple | ||
| 1880 : | weighting of servers. Other properties that can be taken into | ||
| 1881 : | consideration include individual server response time, connection | ||
| 1882 : | count, proximity to client, time zone and various traffic | ||
| 1883 : | characteristics. | ||
| 1884 : | |||
| 1885 : | |||
| 1886 : | \section{Exercises} | ||
| 1887 : | The exercises below are fairly practical, and will give you a feeling | ||
| 1888 : | of the possibilities of high-level load balancing. They focus on | ||
| 1889 : | setup and configuration of DNS and some special HA software. | ||
| 1890 : | |||
| 1891 : | \begin{todo} | ||
| 1892 : | \emph{This exercise is about setting up a dynamically configured DNS | ||
| 1893 : | zone using PowerDNS, and testing the setup.} | ||
| 1894 : | |||
| 1895 : | \begin{itemize} | ||
| 1896 : | \item Skim through the PowerDNS documentation to get an impression of | ||
| 1897 : | the requirements for installation. You will need PowerDNS itself and a | ||
| 1898 : | supported back-end of your choice. For example you can choose MySQL or | ||
| 1899 : | PostgreSQL. All these software suites have ready-to-install packages | ||
| 1900 : | available for most Linux distributions. | ||
| 1901 : | \item Create a database for use with PowerDNS, and populate it with | ||
| 1902 : | the required tables. Alter the default table schema to include a | ||
| 1903 : | weight or preference column. | ||
| 1904 : | \item Design a simple DNS zone containing a cluster of servers in an | ||
| 1905 : | RRset. Use a reasonable number of elements in the set to allow | ||
| 1906 : | testing; five or more is recommended. Implement the zone in the | ||
| 1907 : | database tables. | ||
| 1908 : | \item Set a static preference value for each server. | ||
| 1909 : | \item In the PowerDNS configuration file, set up the \emph{*-query} | ||
| 1910 : | strings. You can experiment with the SQL queries and observe their | ||
| 1911 : | effect. Expressions can include \emph{ORDER BY random()}, \emph{ORDER | ||
| 1912 : | BY \textless col\textgreater}, \emph{LIMIT \textless n\textgreater}, or otherwise any valid SQL supported by | ||
| 1913 : | the back-end. | ||
| 1914 : | \item Start the server and test the configuration by running dig. An | ||
| 1915 : | example of use is \emph{dig @127.0.0.1 a www.test.lan}. | ||
| 1916 : | \end{itemize} | ||
| 1917 : | The following set of tasks is a continuation of the above, but with a | ||
| 1918 : | higher difficulty. | ||
| 1919 : | \begin{itemize} | ||
| 1920 : | \item (Optional) Set up the set of servers in a virtual network, and | ||
| 1921 : | install a web server on them. The servers need not be identical; | ||
| 1922 : | resource variation is interesting for testing, e.g.~CPU power, RAM, | ||
| 1923 : | etc. | ||
| 1924 : | \item (Optional) Create a script on the DNS server that queries the | ||
| 1925 : | load (or other property) of the web servers. Propagate the values into | ||
| 1926 : | the database back-end. Make sure the SQL query strings set up in | ||
| 1927 : | PowerDNS take the preference value into account. | ||
| 1928 : | \item Vary the load (or other property) on the web servers either | ||
| 1929 : | artificially or otherwise, and observe whether the DNS server reacts | ||
| 1930 : | correctly to the measures you impose on the web servers. | ||
| 1931 : | \end{itemize} | ||
| 1932 : | \end{todo} | ||
| 1933 : | |||
| 1934 : | \begin{todo} | ||
| 1935 : | Assuming that the class is divided into groups with two persons in | ||
| 1936 : | each group. This somewhat complex exercise is about configuring a | ||
| 1937 : | fully operating ``global'' network. Depending on the number of groups, | ||
| 1938 : | one or two groups set up a redundant DNS service (two or more DNS | ||
| 1939 : | servers), while the other groups set up sites using the Linux Virtual | ||
| 1940 : | Server and keepalived (or similar) load-balancing implementation. | ||
| 1941 : | \end{todo} | ||
| 1942 : | |||
| 1943 : | |||
| 1944 : | |||
| 1945 : | |||
| 1946 : | \chapter {Server Load Balancing} | ||
| 1947 : | As web pages become more user friendly and interactive we see that | ||
| 1948 : | objects such as pictures, media files, cgi scripts and | ||
| 1949 : | databases are more frequently used. This development causes increased | ||
| 1950 : | stress on the servers due to intensified cpu usage and a growing need | ||
| 1951 : | for bandwidth to serve the content. At the same time users expect low | ||
| 1952 : | latency and high availability. This dilemma can be solved by | ||
| 1953 : | implementing load balancing between servers serving content to the | ||
| 1954 : | clients. Load balancing can provide high availability through | ||
| 1955 : | redundant server solutions, and reduce latency by dividing load. | ||
| 1956 : | |||
| 1957 : | |||
| 1958 : | \section{Overview} | ||
| 1959 : | |||
| 1960 : | In this assignment we will use load balancing to distribute packets | ||
| 1961 : | among a set of equal web servers serving HTTP content. For packet redirection, | ||
| 1962 : | a Nortel Application Switch 2208 will be used, and the servers will be hosted on 6 IBM | ||
| 1963 : | bladeservers. Virtual Lans will be used to separate the external and | ||
| 1964 : | internal traffic, and a separate VLAN should be used for management of | ||
| 1965 : | the switches. | ||
| 1966 : | |||
| 1967 : | When the first part of the assignment is finished, the experimental | ||
| 1968 : | setup, we will look at how the different load balancing algorithms | ||
| 1969 : | compare to each other. | ||
| 1970 : | |||
| 1971 : | \section{\textbf{Part One:} Experimental Setup} | ||
| 1972 : | |||
| 1973 : | In order to perform load balancing we need to configure a topology | ||
| 1974 : | that allows it. In figure \ref{lab-overview} it is used three virtual | ||
| 1975 : | lans to separate network traffic, and make all requests pass through | ||
| 1976 : | the alteon load balancer: | ||
| 1977 : | |||
| 1978 : | \begin{description} | ||
| 1979 : | |||
| 1980 : | \item[VLAN 1 - Administration VLAN (192.168.1/24):] This VLAN is for | ||
| 1981 : | administration purposes only. The network equipment used is configured | ||
| 1982 : | with management interfaces on this VLAN for easy access through the | ||
| 1983 : | network. The baystack switch is configured with management interface | ||
| 1984 : | 192.168.1.2 and the application switch has the address 192.168.1.1. | ||
| 1985 : | Both these units can be accessed through ssh or telnet depending on | ||
| 1986 : | configuration. The default gateway for this vlan is 192.168.1.1, which | ||
| 1987 : | is the interface configured on the application switch. | ||
| 1988 : | |||
| 1989 : | \item[VLAN 10 - Client VLAN (192.168.10/24):] This VLAN is the client | ||
| 1990 : | VLAN. This is the VLAN which gives us access to the network from the | ||
| 1991 : | outside. The machines accessible through internet have two interfaces, | ||
| 1992 : | one of them is located on VLAN 10 with address 192.168.10.2 (see | ||
| 1993 : | figure \ref{lab-overview}) and uses 192.168.10.1 as gateway. | ||
| 1994 : | 192.168.10.1 is configured on the application switch. | ||
| 1995 : | |||
| 1996 : | \item[VLAN 100 - Server VLAN (192.168.100/24):] This VLAN is created | ||
| 1997 : | for the servers, which have ip addresses in the range | ||
| 1998 : | 192.168.100.11-16. They all use 192.168.100.1 as default gateway, this | ||
| 1999 : | address is configured on the application switch. | ||
| 2000 : | \end{description} | ||
| 2001 : | |||
| 2002 : | \begin{figure}[h] | ||
| 2003 : | \centering | ||
| 2004 : | \psfig{file=lab-overview.eps,width=12cm} | ||
| 2005 : | \caption{Experimental Setup: This drawing shows how | ||
| 2006 : | you can setup an environment for doing load balancing in the lab. In | ||
| 2007 : | this setup we have gigabit speeds from the client to the blades.\label{lab-overview}} | ||
| 2008 : | \end{figure} | ||
| 2009 : | |||
| 2010 : | |||
| 2011 : | \subsection{Configuration - Alteon Application Switch} | ||
| 2012 : | You should start to configure the alteon application switch. The | ||
| 2013 : | switch should have a management ip of \textbf{10.0.0.1/24}. It is also | ||
| 2014 : | possible to connect to it using a regular console cable. Below is some | ||
| 2015 : | of the important configuration changes you should make: | ||
| 2016 : | |||
| 2017 : | \begin{enumerate} | ||
| 2018 : | \item Create three vlans; 1,10 and 100, and assign the correct ports. | ||
| 2019 : | Remember to tag the port you | ||
| 2020 : | use as the trunk between the baystack and the alteon application | ||
| 2021 : | switch. | ||
| 2022 : | |||
| 2023 : | \item Create three ip interfaces, one for each vlan, and assign them to | ||
| 2024 : | their correct vlan. | ||
| 2025 : | |||
| 2026 : | \item Create "real" servers, one for each blade. | ||
| 2027 : | |||
| 2028 : | \item Create a group, and then add all the real servers to this group. | ||
| 2029 : | |||
| 2030 : | \item The ports where the blades are attached need to be set as | ||
| 2031 : | "server" port, and the port where you connect your client needs to be | ||
| 2032 : | set as "client" port. If you configure the network as in | ||
| 2033 : | \ref{lab-overview} you will have one port that is both marked as | ||
| 2034 : | "client" and "server" port. | ||
| 2035 : | |||
| 2036 : | \item Create a virtual ip address, this is the address the clients are | ||
| 2037 : | accessing to reach the servers. This address should reside on the 100 | ||
| 2038 : | segment. You also need to enable http traffic for this interface. All | ||
| 2039 : | requests to this address will be load balanced between the real | ||
| 2040 : | servers you have defined. | ||
| 2041 : | |||
| 2042 : | \item Last you need to attach your defined virtual ip to the group | ||
| 2043 : | containing the real servers. | ||
| 2044 : | |||
| 2045 : | \end{enumerate} | ||
| 2046 : | |||
| 2047 : | \noindent \textbf{PS!} Remember to apply and then save your | ||
| 2048 : | configuration. Backup can also be made to a tftp server.\\\\ | ||
| 2049 : | \medskip | ||
| 2050 : | \noindent \textbf{PS!} Documentation on how to configure load | ||
| 2051 : | balancing in more detail can be found here: | ||
| 2052 : | \url{http://www.nortel.com}, look for documents regarding the 2208 | ||
| 2053 : | application switch. | ||
| 2054 : | |||
| 2055 : | \subsection{Baystack Switch} | ||
| 2056 : | The baystack switch can be configured either through the network, if | ||
| 2057 : | it has a management ip, or through the console. | ||
| 2058 : | |||
| 2059 : | \begin{enumerate} | ||
| 2060 : | \item Begin by creating the same three vlans you created on the | ||
| 2061 : | application switch. Assign the correct ports to the correct vlans. The | ||
| 2062 : | vlan trunk going to the application switch need to be member of all | ||
| 2063 : | three vlans, and it needs to be tagged. | ||
| 2064 : | |||
| 2065 : | \item Set a management ip in the vlan 1 segment. | ||
| 2066 : | \item Set the port connecting the baystack and the application switch | ||
| 2067 : | to tagged, and enable all three vlans to it. | ||
| 2068 : | \item Assign vlans to the other ports according to where you connected | ||
| 2069 : | the other equipment (blades, client). | ||
| 2070 : | \end{enumerate} | ||
| 2071 : | |||
| 2072 : | \subsection{IBM blades} | ||
| 2073 : | The blades should all have the same linux distro installed, as well as | ||
| 2074 : | apache. How you wish to configure apache is up to yourself, but it | ||
| 2075 : | should be well documented. The blades are capable of handling a lot of | ||
| 2076 : | requests per second, so it might be wise to install php as well. This | ||
| 2077 : | way you can create a cpu intensive php script to use when generating | ||
| 2078 : | traffic, this can help you eliminate the possibility of your | ||
| 2079 : | client being the bottleneck. You also need to configure addresses on | ||
| 2080 : | each of the blades that corresponds to the "real" servers you defined | ||
| 2081 : | on the application switch. And you should use the ip interface | ||
| 2082 : | attached to VLAN 100 on the application switch as the default gateway. | ||
| 2083 : | |||
| 2084 : | \subsection{Client} | ||
| 2085 : | On this machine you need to install or create a program to generate | ||
| 2086 : | traffic against the VIP address you created on the application switch. | ||
| 2087 : | You also need to configure the interface connected to the baystack | ||
| 2088 : | with an ip address in the vlan 10 segment. The default gateway should | ||
| 2089 : | be set to the ip interface you attached to vlan 10 on the application | ||
| 2090 : | switch. You also need to add routes to all the other vlans, using the | ||
| 2091 : | same gateway as your default gateway. | ||
| 2092 : | |||
| 2093 : | \section{\textbf{Part Two:} Testing Performance} | ||
| 2094 : | In part two of the assignment you can choose between three different | ||
| 2095 : | testing scenarios: | ||
| 2096 : | |||
| 2097 : | \subsection{Benchmarking Algorithms} | ||
| 2098 : | To be able to compare the efficiency of different load balancing | ||
| 2099 : | algorithms, you will perform a series of tests with each of the | ||
| 2100 : | available algorithms and | ||
| 2101 : | compare the results. Look at properties such as response time | ||
| 2102 : | and load distribution. | ||
| 2103 : | |||
| 2104 : | \subsection{Scalability} | ||
| 2105 : | In these experiments you should test how performance | ||
| 2106 : | scales when adding additional servers. Run tests with increasing | ||
| 2107 : | rates, and then plot the performance. Try to see if | ||
| 2108 : | there is a linear increase in performance when adding blades. | ||
| 2109 : | |||
| 2110 : | \subsection{Experiment vs. Queueing models (HARD)} | ||
| 2111 : | In these experiments you should look at how the blades perform compared | ||
| 2112 : | to $M/M/k$ and $M/M/1^k$ queueing models. | ||
| 2113 : | |||
| 2114 : | \section{Important Considerations} | ||
| 2115 : | When doing measurements on the performance of load balancing it is important that | ||
| 2116 : | we are aware of the constraints. Some of the constraints that apply to | ||
| 2117 : | this experiment are listed below, these factors should be considered | ||
| 2118 : | when analyzing the results: | ||
| 2119 : | |||
| 2120 : | \begin{description} | ||
| 2121 : | \item[Processing power:] Each of the blades in use have limited | ||
| 2122 : | processing power. When this limit is exceeded it might be difficult to | ||
| 2123 : | acquire accurate measurements from the blades. | ||
| 2124 : | |||
| 2125 : | \item[Network Bandwith:] All blades are connected to the network with | ||
| 2126 : | GigaBit (GB) interfaces, but since all blades are connected through | ||
| 2127 : | one | ||
| 2128 : | CAT6 Twisted Pair (TP) cable, they share 1GB of bandwidth. | ||
| 2129 : | |||
| 2130 : | \item[Load Balancer:] The load balancer must process packets, and the | ||
| 2131 : | amount of processing depends on topology and algorithms used. The load | ||
| 2132 : | balancer could become a limitation in extreme cases. | ||
| 2133 : | |||
| 2134 : | \item[Apache Webserver:] The apache webserver is configured to spawn a | ||
| 2135 : | limited number of child processes to handle incoming requests. When this limit | ||
| 2136 : | is reached, it starts to put requests in a queue, and when the queue is full the | ||
| 2137 : | server do not accept more incoming requests. | ||
| 2138 : | |||
| 2139 : | \item[Client limitations:] Our traffic generator act as a client | ||
| 2140 : | generating traffic towards the webservers. The client can only | ||
| 2141 : | generate a limited amount of connections based on a filedescriptor | ||
| 2142 : | value (each connection needs a filedescriptor). | ||
| 2143 : | |||
| 2144 : | \end{description} | ||
| 2145 : | |||
| 2146 : | \part{Clusters} | ||
| 2147 : | |||
| 2148 : | |||
| 2149 : | |||
| 2150 : | \include{cluster-gen-tore} | ||
| 2151 : | |||
| 2152 : | |||
| 2153 : | \bibliography{LISAbib} | ||
| 2154 : | |||
| 2155 : | \end{document} |
| Administrator | ViewVC Help |
| Powered by ViewVC 1.0.3 |