Parent Directory
| 
Revision Log
Revision 1 - (view) (download) (as text)
| 1 : | mark | 1 | \documentclass[a4paper,10pt]{article} |
| 2 : | \usepackage{times} % -- good | ||
| 3 : | \usepackage{fancyheadings} | ||
| 4 : | \usepackage{tocloft} | ||
| 5 : | |||
| 6 : | |||
| 7 : | \usepackage{a4} | ||
| 8 : | %\usepackage{shadethm} | ||
| 9 : | \pagestyle{fancyplain} | ||
| 10 : | \lhead[\fancyplain{}{}]{\fancyplain{}{\rightmark}} | ||
| 11 : | \rhead[\fancyplain{}{\leftmark}]{\fancyplain{}{}} | ||
| 12 : | |||
| 13 : | %\textheight 20cm | ||
| 14 : | \textwidth 15.5cm | ||
| 15 : | \oddsidemargin -20pt | ||
| 16 : | \evensidemargin -20pt | ||
| 17 : | |||
| 18 : | |||
| 19 : | \usepackage{url} | ||
| 20 : | \usepackage{tabularx} | ||
| 21 : | |||
| 22 : | \usepackage{fancybox} | ||
| 23 : | \usepackage{fancyvrb} | ||
| 24 : | |||
| 25 : | \urlstyle{rm} | ||
| 26 : | %\def\UrlFont{\em} | ||
| 27 : | |||
| 28 : | \newenvironment{code}% | ||
| 29 : | {\begin{Sbox}\begin{minipage}{.9\textwidth}\scriptsize}% | ||
| 30 : | {\end{minipage}\end{Sbox}% | ||
| 31 : | \begin{center}\setlength{\fboxsep}{2pt}\setlength{\fboxrule}{1pt}% | ||
| 32 : | \fbox{\TheSbox}\end{center}} | ||
| 33 : | |||
| 34 : | \begin{document} | ||
| 35 : | \newtheorem{example}{Example} | ||
| 36 : | \newtheorem{question}{Question} | ||
| 37 : | |||
| 38 : | \section{Load Balancing} | ||
| 39 : | What do you do when your web server is overloaded with heavy requests? | ||
| 40 : | One suggestion is to add hardware upgrades to the server, such as a | ||
| 41 : | stronger processor and more memory. Another idea is to support the web | ||
| 42 : | site with several servers. Using some kind of request scheduling | ||
| 43 : | mechanism, it is then possible to share the load among the available | ||
| 44 : | servers. This example, though trivial, outlines the essence of load | ||
| 45 : | balancing: Sharing an imposed workload between several ordinary and | ||
| 46 : | relatively inexpensive servers, as opposed to handling the entire load | ||
| 47 : | on a single high-end node. In simplified queueing theory terms, we are | ||
| 48 : | moving from $M/M/1$ to $M/M/k$ or $M/M/1^k$ queues. In other words, we | ||
| 49 : | substitute the single server setup with more servers or service | ||
| 50 : | channels. | ||
| 51 : | |||
| 52 : | Sharing load between servers not only has direct economical effects, | ||
| 53 : | it often brings about welcome side effects. One such benefit is the | ||
| 54 : | level of redundancy implied by running a load balanced service. Say | ||
| 55 : | for example that you serve a corporate web site between ten servers. | ||
| 56 : | Then you might afford to lose one or more servers without severely | ||
| 57 : | impacting the browsing experience for customers. This approach can | ||
| 58 : | easily alleviate software or hardware faults in single nodes. It | ||
| 59 : | should be noted, however, that a proper redundant system requires | ||
| 60 : | redundancy in all links of the chain. | ||
| 61 : | |||
| 62 : | %On the issue of manageability, keeping track of software and general | ||
| 63 : | %operation of a computer cluster can be considered a drawback compared | ||
| 64 : | %to the monolithic approach. However, with the right tools and | ||
| 65 : | %configuration, difficulty of management need not turn out to be the | ||
| 66 : | %case at all. | ||
| 67 : | |||
| 68 : | It is apparent that the difference between lower and higher levels of | ||
| 69 : | load balancing is directly related to granularity and scalability, | ||
| 70 : | both when it comes to data handling and for manageability. For | ||
| 71 : | example, in a multi-CPU system the task scheduler in the operating | ||
| 72 : | system organises processes and threads so that they are load-balanced | ||
| 73 : | on the available CPU resources. On this scale we deal with units of | ||
| 74 : | CPU cycles and microsecond deltas, which are not directly relevant | ||
| 75 : | when we talk about server or network load balancing. However, if we | ||
| 76 : | zoom out several orders of magnitude and look at network topologies, | ||
| 77 : | we can typically operate with units of packets and bytes, and typical | ||
| 78 : | deltas of seconds to hours. This is where load balancing becomes more | ||
| 79 : | interesting and manageable in system administration. Towards the | ||
| 80 : | topmost levels are considerations such as geographic proximity, | ||
| 81 : | routing mechanisms and service level agreements. These are interesting | ||
| 82 : | because they move away from details and allow us to get a handle on | ||
| 83 : | the big picture of large-scale network traffic and do load balancing | ||
| 84 : | of that very same traffic. This implies that we deal with trends and | ||
| 85 : | analysis of large amounts of data over very long time spans, typically | ||
| 86 : | hours and days, to months and even years. | ||
| 87 : | |||
| 88 : | \section{DNS-based Load Balancing} | ||
| 89 : | The Domain Name System (DNS) is the global name lookup service in the | ||
| 90 : | Internet. It is a distributed, hierarchical and redundant database | ||
| 91 : | running on thousands of servers worldwide, each responsible for one or | ||
| 92 : | more DNS zones. When a client issues a request for a URL, the browser | ||
| 93 : | will first try to resolve the hostname in the URL into an IP address, | ||
| 94 : | so that it knows where to send the request. This is where it is | ||
| 95 : | possible for a DNS server to influence the outcome of a query, | ||
| 96 : | effectively directing the client to a desired or lightly loaded site. | ||
| 97 : | |||
| 98 : | While DNS in itself was not designed particularly with the | ||
| 99 : | requirements of load balancing in mind, it does provide a limited set | ||
| 100 : | of options to facilitate simple balancing schemes. However, together | ||
| 101 : | with dynamical back-ends, DNS can be a well suited front-end to robust | ||
| 102 : | load balancing mechanisms. The strength of using DNS is its almost | ||
| 103 : | universal deployment: Practically every node connected to an IP | ||
| 104 : | network is capable of DNS resolving. This makes DNS one of the most | ||
| 105 : | desirable \emph{interfaces} for global load sharing implementations. | ||
| 106 : | Dynamical back-ends are discussed later. | ||
| 107 : | |||
| 108 : | To give an insight into how nameservers respond to queries, consider | ||
| 109 : | the listing below, showing what a client would receive (per July 2006) | ||
| 110 : | when asking for the IP address of the cnn.com host. This particular | ||
| 111 : | output is from the `dig' application, and shows three main sections: | ||
| 112 : | The question section echoes the request information; the answer | ||
| 113 : | section shows the matching records (if any); the authority section | ||
| 114 : | lists the authoritative nameservers for the domain. | ||
| 115 : | |||
| 116 : | \setlength{\fboxsep}{0pt} | ||
| 117 : | \begin{code} | ||
| 118 : | \fvset{numbers=left,commandchars=\\\{\}} | ||
| 119 : | \begin{Verbatim} | ||
| 120 : | ;; QUESTION SECTION: | ||
| 121 : | ;cnn.com. IN A | ||
| 122 : | |||
| 123 : | ;; ANSWER SECTION: | ||
| 124 : | cnn.com. 300 IN A 64.236.29.120 ;This | ||
| 125 : | cnn.com. 300 IN A 64.236.16.20 ;is | ||
| 126 : | cnn.com. 300 IN A 64.236.16.52 ;an | ||
| 127 : | cnn.com. 300 IN A 64.236.16.84 ;RRset | ||
| 128 : | cnn.com. 300 IN A 64.236.16.116 ;with | ||
| 129 : | cnn.com. 300 IN A 64.236.24.12 ;eight | ||
| 130 : | cnn.com. 300 IN A 64.236.24.20 ;web | ||
| 131 : | cnn.com. 300 IN A 64.236.24.28 ;servers | ||
| 132 : | |||
| 133 : | ;; AUTHORITY SECTION: | ||
| 134 : | cnn.com. 600 IN NS twdns-04.ns.aol.com. | ||
| 135 : | cnn.com. 600 IN NS twdns-01.ns.aol.com. | ||
| 136 : | cnn.com. 600 IN NS twdns-02.ns.aol.com. | ||
| 137 : | cnn.com. 600 IN NS twdns-03.ns.aol.com. | ||
| 138 : | \end{Verbatim} | ||
| 139 : | \end{code} | ||
| 140 : | |||
| 141 : | There are a few interesting properties of the DNS response in the | ||
| 142 : | listing. First of all, it is clear that the nameservers return | ||
| 143 : | multiple A-records for the cnn.com hostname -- this is known as a | ||
| 144 : | resource record set (RRset). It is difficult to determine whether | ||
| 145 : | there is a complex algorithm behind the result or if it is simply | ||
| 146 : | a random permutation of the available servers. Because ordering of | ||
| 147 : | RRsets is never guaranteed, it is likely that the DNS servers use | ||
| 148 : | random scheduling. In other words, they do not map server preference | ||
| 149 : | to the RRset ordering, such as placing the most preferred server on | ||
| 150 : | top of the list. | ||
| 151 : | |||
| 152 : | Another point to notice is the time-to-live (TTL) values in the second | ||
| 153 : | column indicate that the returned records have a very short caching | ||
| 154 : | lifetime, 5 and 10 minutes for the A and NS records, respectively. A | ||
| 155 : | low TTL ensures that DNS servers are queried often, and are thus | ||
| 156 : | able to influence the answer over relatively small time intervals, | ||
| 157 : | e.g.~removing dead servers from the RRset. Low TTLs come at the cost | ||
| 158 : | of higher frequency of queries, which can add a considerable delay to | ||
| 159 : | the total page response time. | ||
| 160 : | |||
| 161 : | \section{Configuring BIND} | ||
| 162 : | A simple setup of DNS load balancing in the BIND software takes very | ||
| 163 : | little effort. The following listing is an example taken from a normal | ||
| 164 : | BIND zone file with standard syntax, and it describes the fictional | ||
| 165 : | \url{test.lan} zone: | ||
| 166 : | \setlength{\fboxsep}{0pt} | ||
| 167 : | \begin{code} | ||
| 168 : | \fvset{numbers=left,commandchars=\\\{\}} | ||
| 169 : | \begin{Verbatim} | ||
| 170 : | \$TTL 60 | ||
| 171 : | test.lan. IN SOA ns.test.lan. hostmaster.test.lan. ( | ||
| 172 : | 1 ; Serial | ||
| 173 : | 604800 ; Refresh | ||
| 174 : | 86400 ; Retry | ||
| 175 : | 2419200 ; Expire | ||
| 176 : | 604800 ) ; Negative Cache TTL | ||
| 177 : | ; | ||
| 178 : | test.lan. IN NS ns.test.lan. ; NameServer record. | ||
| 179 : | ns IN A 10.0.0.1 ; A-record for nameserver. | ||
| 180 : | www IN A 10.0.0.13 | ||
| 181 : | www IN A 10.0.0.11 | ||
| 182 : | www IN A 10.0.0.14 | ||
| 183 : | www IN A 10.0.0.16 | ||
| 184 : | www IN A 10.0.0.15 | ||
| 185 : | www IN A 10.0.0.12 | ||
| 186 : | \end{Verbatim} | ||
| 187 : | \end{code} | ||
| 188 : | The first line defines the time-to-live value to use for all the | ||
| 189 : | records contained within the zonefile. Next, on lines 2 through 7, is | ||
| 190 : | the \emph{start of authority} record, which describes behaviour of | ||
| 191 : | slave nameservers. The relevant records are the six A-records that | ||
| 192 : | make up the RRset for the \url{www.test.lan} hostname. As a result, | ||
| 193 : | any lookup for the A-record for \url{www.test.lan} would return the | ||
| 194 : | entire set of addresses to the querying client. The default scheduling | ||
| 195 : | mechanism in BIND has been known to vary with releases, but is | ||
| 196 : | typically set to either random or round-robin with an arbitrary | ||
| 197 : | starting order. It can be configured in the \emph{named} configuration | ||
| 198 : | options, using the directive ``rr-set ordering''. | ||
| 199 : | |||
| 200 : | \section{Dynamical DNS Zones} | ||
| 201 : | The current (July 2006) DNS scheme deployed on \url{amazon.com} offers | ||
| 202 : | another approach to load sharing. When querying the DNS servers for | ||
| 203 : | the A record of \url{www.amazon.com}, only one address is returned. | ||
| 204 : | Subsequent queries show that the server returns a single IP address | ||
| 205 : | out of a small set. The effects of this approach are twofold. On one | ||
| 206 : | side it lets the DNS servers decide which server the client connects | ||
| 207 : | to -- i.e. the load sharing is very much more controlled than in the | ||
| 208 : | previous CNN example. On the other hand this solution introduces a | ||
| 209 : | single point of failure for short periods of time: The client is given | ||
| 210 : | only one point of correspondence; should it fail, the client must give | ||
| 211 : | up or wait for the TTL to expire. By then, Amazon might have lost | ||
| 212 : | their customer. | ||
| 213 : | |||
| 214 : | Is there a middle ground here, that is, a compromise between load | ||
| 215 : | sharing control and availability? Such a compromise could be | ||
| 216 : | implemented by having the DNS servers reply with a \emph{subset} of | ||
| 217 : | the full list of servers. It can be argued that this solution provides | ||
| 218 : | both the control and availability needed for a high-volume site. | ||
| 219 : | |||
| 220 : | When implementing dynamical DNS zones, there is some choice of software. | ||
| 221 : | For many years the PowerDNS package has been a viable alternative to | ||
| 222 : | BIND. PowerDNS is an authoritative-only DNS server with a very | ||
| 223 : | flexible back-end architecture: It supports loading zones from plain | ||
| 224 : | BIND-style zone files, a variety of SQL databases, LDAP, etc. Examples | ||
| 225 : | of use include setting server preference using a weight factor, or | ||
| 226 : | limiting the result to a subset of the matching A-records. | ||
| 227 : | |||
| 228 : | On a side note, the latest version of BIND, v9.4, features the DLZ | ||
| 229 : | extension. DLZ, dynamically loadable zones, gives BIND much of the | ||
| 230 : | same back-end capabilities as PowerDNS when it comes to serving | ||
| 231 : | dynamical content. | ||
| 232 : | |||
| 233 : | \section{Configuring PowerDNS} | ||
| 234 : | Configuring PowerDNS zones is done by populating a database of choice | ||
| 235 : | with the zone information, and then specifying the SQL query needed to | ||
| 236 : | fetch the data in the configuration file. An example of a query from a | ||
| 237 : | populated database is shown below. Note that it contains mostly the | ||
| 238 : | same information as the static zone file (though the SOA content is | ||
| 239 : | truncated): | ||
| 240 : | \setlength{\fboxsep}{0pt} | ||
| 241 : | \begin{code} | ||
| 242 : | \fvset{numbers=left} | ||
| 243 : | \begin{Verbatim} | ||
| 244 : | # select id,name,type,content,ttl,pref from records; | ||
| 245 : | |||
| 246 : | id | name | type | content | ttl | pref | ||
| 247 : | ----+--------------+------+---------------------+------+------ | ||
| 248 : | 1 | test.lan | SOA | ns.test.lan hostmast| 86400| | ||
| 249 : | 2 | test.lan | NS | ns.test.lan | 86400| | ||
| 250 : | 9 | ns.test.lan | A | 10.0.0.1 | 86400| | ||
| 251 : | 3 | www.test.lan | A | 10.0.0.11 | 30| 10 | ||
| 252 : | 4 | www.test.lan | A | 10.0.0.12 | 30| 15 | ||
| 253 : | 5 | www.test.lan | A | 10.0.0.13 | 30| 60 | ||
| 254 : | 6 | www.test.lan | A | 10.0.0.14 | 30| 45 | ||
| 255 : | 7 | www.test.lan | A | 10.0.0.15 | 30| 100 | ||
| 256 : | 8 | www.test.lan | A | 10.0.0.16 | 30| 30 | ||
| 257 : | \end{Verbatim} | ||
| 258 : | \end{code} | ||
| 259 : | By appending the ``ORDER BY random() LIMIT 1'' expression to the SQL | ||
| 260 : | query, only one random record is returned (the Amazon case). | ||
| 261 : | Similarly, ``ORDER BY pref LIMIT 3'' would return the three most | ||
| 262 : | preferable servers. If the \emph{pref} value is static, the three | ||
| 263 : | least preferable servers would never be used. This can be solved by | ||
| 264 : | making the pref value dynamically updated using a simple health-check | ||
| 265 : | system. An example of such a system would be monitoring server load in | ||
| 266 : | short intervals, and updating the pref value to reflect the current | ||
| 267 : | load scenario. | ||
| 268 : | |||
| 269 : | The practical uses of dynamical zones are many and vary beyond simple | ||
| 270 : | weighting of servers. Other properties that can be taken into | ||
| 271 : | consideration include individual server response time, connection | ||
| 272 : | count, proximity to client, time zone and various traffic | ||
| 273 : | characteristics. | ||
| 274 : | |||
| 275 : | |||
| 276 : | \section{Exercises} | ||
| 277 : | The exercises below are fairly practical, and will give you a feeling | ||
| 278 : | of the possibilities of high-level load balancing. They focus on | ||
| 279 : | setup and configuration of DNS and some special HA software. | ||
| 280 : | |||
| 281 : | \subsection{Exercise One (simple)} | ||
| 282 : | \emph{This exercise is about setting up a dynamically configured DNS | ||
| 283 : | zone using PowerDNS, and testing the setup.} | ||
| 284 : | |||
| 285 : | \begin{itemize} | ||
| 286 : | \item Skim through the PowerDNS documentation to get an impression of | ||
| 287 : | the requirements for installation. You will need PowerDNS itself and a | ||
| 288 : | supported back-end of your choice. For example you can choose MySQL or | ||
| 289 : | PostgreSQL. All these software suites have ready-to-install packages | ||
| 290 : | available for most Linux distributions. | ||
| 291 : | \item Create a database for use with PowerDNS, and populate it with | ||
| 292 : | the required tables. Alter the default table schema to include a | ||
| 293 : | weight or preference column. | ||
| 294 : | \item Design a simple DNS zone containing a cluster of servers in an | ||
| 295 : | RRset. Use a reasonable number of elements in the set to allow | ||
| 296 : | testing; five or more is recommended. Implement the zone in the | ||
| 297 : | database tables. | ||
| 298 : | \item Set a static preference value for each server. | ||
| 299 : | \item In the PowerDNS configuration file, set up the \emph{*-query} | ||
| 300 : | strings. You can experiment with the SQL queries and observe their | ||
| 301 : | effect. Expressions can include \emph{ORDER BY random()}, \emph{ORDER | ||
| 302 : | BY \textless col\textgreater}, \emph{LIMIT \textless n\textgreater}, or otherwise any valid SQL supported by | ||
| 303 : | the back-end. | ||
| 304 : | \item Start the server and test the configuration by running dig. An | ||
| 305 : | example of use is \emph{dig @127.0.0.1 a www.test.lan}. | ||
| 306 : | \end{itemize} | ||
| 307 : | The following set of tasks is a continuation of the above, but with a | ||
| 308 : | higher difficulty. | ||
| 309 : | \begin{itemize} | ||
| 310 : | \item (Optional) Set up the set of servers in a virtual network, and | ||
| 311 : | install a web server on them. The servers need not be identical; | ||
| 312 : | resource variation is interesting for testing, e.g.~CPU power, RAM, | ||
| 313 : | etc. | ||
| 314 : | \item (Optional) Create a script on the DNS server that queries the | ||
| 315 : | load (or other property) of the web servers. Propagate the values into | ||
| 316 : | the database back-end. Make sure the SQL query strings set up in | ||
| 317 : | PowerDNS take the preference value into account. | ||
| 318 : | \item Vary the load (or other property) on the web servers either | ||
| 319 : | artificially or otherwise, and observe whether the DNS server reacts | ||
| 320 : | correctly to the measures you impose on the web servers. | ||
| 321 : | \end{itemize} | ||
| 322 : | |||
| 323 : | |||
| 324 : | \subsection{Exercise Two (complex)} | ||
| 325 : | \emph{Assuming that the class is divided into groups with two persons in | ||
| 326 : | each group. This somewhat complex exercise is about configuring a | ||
| 327 : | fully operating ``global'' network. Depending on the number of groups, | ||
| 328 : | one or two groups set up a redundant DNS service (two or more DNS | ||
| 329 : | servers), while the other groups set up sites using the Linux Virtual | ||
| 330 : | Server and keepalived (or similar) load-balancing implementation.} | ||
| 331 : | |||
| 332 : | \end{document} |
| Administrator | ViewVC Help |
| Powered by ViewVC 1.0.3 |