{"id":318,"date":"2014-03-26T20:57:58","date_gmt":"2014-03-26T20:57:58","guid":{"rendered":"http:\/\/blogs.nd.edu\/devops\/?p=318"},"modified":"2014-03-27T15:42:57","modified_gmt":"2014-03-27T15:42:57","slug":"why-we-test","status":"publish","type":"post","link":"https:\/\/sites.nd.edu\/devops\/2014\/03\/26\/why-we-test\/","title":{"rendered":"Why We Test"},"content":{"rendered":"

Despite design principles, despite best intentions, things can go wrong. \u00a0This is true when all of the design parameters are quantifiable, as exhibited by this spectacular engine failure:\"F1EngineFail\"<\/a>\u00a0What happens when some of the design constraints are not fully understood? \u00a0Specifically, consider the following excerpt taken from Amazon’s page detailing instance specifications:<\/p>\n\n\n\n
Instance Family<\/td>\nInstance Type<\/td>\nProcessor Arch<\/td>\nvCPU<\/td>\nECU<\/td>\nMemory (GiB)<\/td>\nInstance Storage (GB)<\/td>\nEBS-optimized Available<\/td>\nNetwork Performance<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
\n\n\n\n\n\n\n\n\n\n
Storage optimized<\/td>\ni2.xlarge<\/td>\n64-bit<\/td>\n4<\/td>\n14<\/td>\n30.5<\/td>\n1 x 800 SSD<\/td>\nYes<\/td>\nModerate<\/td>\n<\/tr>\n
Storage optimized<\/td>\ni2.2xlarge<\/td>\n64-bit<\/td>\n8<\/td>\n27<\/td>\n61<\/td>\n2 x 800 SSD<\/td>\nYes<\/td>\nHigh<\/td>\n<\/tr>\n
Storage optimized<\/td>\ni2.4xlarge<\/td>\n64-bit<\/td>\n16<\/td>\n53<\/td>\n122<\/td>\n4 x 800 SSD<\/td>\nYes<\/td>\nHigh<\/td>\n<\/tr>\n
Storage optimized<\/td>\ni2.8xlarge<\/td>\n64-bit<\/td>\n32<\/td>\n104<\/td>\n244<\/td>\n8 x 800 SSD<\/td>\n–<\/td>\n10 Gigabit*4<\/td>\n<\/tr>\n
Storage optimized<\/td>\nhs1.8xlarge<\/td>\n64-bit<\/td>\n16<\/td>\n35<\/td>\n117<\/td>\n24 x 2,048*3<\/td>\n–<\/td>\n10 Gigabit*4<\/td>\n<\/tr>\n
Storage optimized<\/td>\nhi1.4xlarge<\/td>\n64-bit<\/td>\n16<\/td>\n35<\/td>\n60.5<\/td>\n2 x 1,024
\nSSD*2<\/td>\n		–<\/td>\n10 Gigabit*4<\/td>\n<\/tr>\n
Micro instances<\/td>\nt1.micro<\/td>\n32-bit or
\n64-bit<\/td>\n		1<\/td>\nVariable*5<\/td>\n0.615<\/td>\nEBS only<\/td>\n–<\/td>\nVery Low<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n
10 Gigabit is a measure we can understand. \u00a0But what does Moderate mean? \u00a0How does it compare to High, or Very Low? \u00a0It is possible to find out, but only if we test.<\/div>\n

According to Amazon’s page on instance types<\/a>\u00a0(emphasis added):<\/p>\n

Amazon EC2 allows you to provision a variety of instances types, which provide different combinations of CPU, memory, disk, and networking. Launching new instances and running tests in parallel is easy, and we recommend measuring the performance of applications to identify appropriate instance types and validate application architecture<\/strong>. We also recommend rigorous load\/scale testing to ensure that your applications can scale as you intend.<\/p><\/blockquote>\n

Using LoadUIWeb<\/a> by Smartbear<\/a>, we were able to take a given workload and run it on different instance types to gain a better understanding of the performance differences. \u00a0Running 100 simultaneous virtual users with a 9 millisecond think time between pages, we ran against an M1.small and an M1.medium. \u00a0Here is are some pictures to illustrate what we discovered:<\/p>\n

M1.small performance test output:<\/p>\n

\"Unknown-2\"<\/a>\"Small\"<\/a><\/p>\n

M1.medium performance test output:<\/p>\n

\"Unknown-1\"<\/a><\/p>\n

\"Medium\"<\/a><\/p>\n

 <\/p>\n

Looking at the numbers, we see that the the response time of the small is roughly twice that of the medium. \u00a0As neither CPU, memory, or disk activity were constraints, this serves as a warning to us – the quality of service limitations placed on different instance types must be taken into consideration when doing instance size analysis.<\/p>\n

When in doubt, test it out!<\/p>\n","protected":false},"excerpt":{"rendered":"

Despite design principles, despite best intentions, things can go wrong. \u00a0This is true when all of the design parameters are quantifiable, as exhibited by this spectacular engine failure:\u00a0What happens when some of the design constraints are not fully understood? \u00a0Specifically, … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":1551,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[65088,65095],"tags":[],"class_list":["post-318","post","type-post","status-publish","format-standard","hentry","category-aws","category-cloud-infrastructure"],"_links":{"self":[{"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/posts\/318","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/users\/1551"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/comments?post=318"}],"version-history":[{"count":5,"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/posts\/318\/revisions"}],"predecessor-version":[{"id":330,"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/posts\/318\/revisions\/330"}],"wp:attachment":[{"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/media?parent=318"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/categories?post=318"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.nd.edu\/devops\/wp-json\/wp\/v2\/tags?post=318"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}