{"id":320,"date":"2018-02-22T19:30:37","date_gmt":"2018-02-22T18:30:37","guid":{"rendered":"http:\/\/vmware.nicolas-frey.de\/blog\/?p=320"},"modified":"2019-01-05T09:17:49","modified_gmt":"2019-01-05T08:17:49","slug":"vmware-vsan-series-architecture","status":"publish","type":"post","link":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/","title":{"rendered":"VMWare vSAN Series – Architecture"},"content":{"rendered":"

Hi everybody,<\/p>\n

in reference to my prior article\u00a0vSAN – High Available Solution<\/a>\u00a0this article will explain the basic architecture and give you some insight into the technical details.<\/p>\n

<\/p>\n

vSAN Basic Architecture<\/h2>\n

There are several different approaches to vSAN cluster. The decision has to be made stretched\/non stretched and direct connect\/multi node based on the requirements such as availability\/performance\/scaleability<\/p>\n

Stretched:<\/strong> at least two nodes are stretched over two sites; additional requirements – 10Gbit and <5ms rtt (round trip time) – layer 3 supported; witness required<\/p>\n

Non Stretched:<\/strong> all nodes are on the same site<\/p>\n

Direct Connect:<\/strong><\/em> just two nodes without redundant ethernet switches, witness required<\/p>\n

Multi node:<\/strong><\/em> (redundant) 10 Gbit infrastructure required; up to 64 nodes in one cluster<\/p>\n

Irrespective which deployment type you choose, every node has in common that it contributes local CPU \/ RAM \/ Network \/ Storage resources to the attached cluster.<\/p>\n

It is possible to add nodes to the cluster without local storage, which just offer computing.<\/p>\n

Network Storage Backend<\/h2>\n

Rule of thumb is that in hybrid configuration 1 Gbit is supported but not recommended.\u00a0 Opposite is all-flash where 10 Gbit is mandatory. If you use PCIe NVMe card which provide transfer rates over 3500 MB\/s you should consider 40 Gbit or even 100 Gbit. Which is at least cheap in 2 node direct connect configurations because you wont need that expensive switch infrastructure.<\/p>\n

To deliver the goods we need at least one more VMKernel for the vSAN traffic.<\/p>\n

If you need to use a Witness you need to tag you management VMKernel with Witness traffic. So far this is just possible over the command line:<\/p>\n

esxcli vsan network ip add -i vmk0 -T=witness<\/strong><\/p>\n

Internal Disk Groups<\/h2>\n

Every disc group consists of exactly one flash caching device (SAS,SATA or NVME) for read caching and write buffering operations – Caching Tier<\/strong><\/p>\n

… and up to 7 flash or spinning disks which offer storage capacity – Storage Tier<\/strong><\/p>\n

Important is that in all-flash<\/strong> configurations the caching tier is 100% write buffer.<\/p>\n

In hybrid<\/strong> configurations (capacity tier is HDD) the caching tier is 30% write buffer and 70% read cache.<\/p>\n

\"\"<\/a><\/p>\n

After creation its formatted in VMFS-L and consequently every Disk group is now a silo for vSAN objects.<\/p>\n

In addition the max. amount of disk groups per host is five.<\/p>\n

vSAN Datastore and Storage Policy<\/h2>\n

In order to create a vSAN Datastore you need existing disk groups which will be aggregated together and compose the vSAN Datastore.<\/p>\n

Therein objects can be saved fulfilling the requirements which will be mapped in the storage policy.<\/p>\n

\"\"<\/a><\/p>\n

So every virtual machine is attached to a storage policy which contains some basic and optional advanced configuration parameters for storage performance and redundancy.<\/p>\n

There a two key parameters which are mandatory in the storage policy.<\/p>\n

Stripe width<\/strong>\u00a0(SW)<\/strong> – object placement striped on different physical disks in order to adapt to performance requirements. Objects sized above 255 GByte are divided automatically into more stripes.<\/p>\n

Faults to tolerate (FTT) <\/strong>– compensation of failing nodes, fault to tolerate = 1 means all object information are at least saved into two different disk groups on different hosts. In conclusion you get availability at the cost of storage when you increase this parameter<\/p>\n

Additional rules can be defined like “flash read cache reservation” or “IOPS limit for object”<\/p>\n

Example vSAN Default Storage Policy:<\/p>\n

\"\"<\/a><\/p>\n

Best regards<\/p>\n

Nicolas Frey<\/p>\n","protected":false},"excerpt":{"rendered":"

Hi everybody, in reference to my prior article\u00a0vSAN – High Available Solution\u00a0this article will explain the basic architecture and give you some insight into the technical details.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[37],"tags":[25,42,8,41],"class_list":["post-320","post","type-post","status-publish","format-standard","hentry","category-vsan","tag-san","tag-software-defined-storage","tag-vmware","tag-vsan"],"yoast_head":"\nVMWare vSAN Series - Architecture - My Experience With VMware Virtualization<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"VMWare vSAN Series - Architecture - My Experience With VMware Virtualization\" \/>\n<meta property=\"og:description\" content=\"Hi everybody, in reference to my prior article\u00a0vSAN – High Available Solution\u00a0this article will explain the basic architecture and give you some insight into the technical details.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/\" \/>\n<meta property=\"og:site_name\" content=\"My Experience With VMware Virtualization\" \/>\n<meta property=\"article:published_time\" content=\"2018-02-22T18:30:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-01-05T08:17:49+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/vmware.nicolas-frey.de\/blog\/wp-content\/uploads\/2018\/02\/VSAN_Caching_Algo.jpg\" \/>\n<meta name=\"author\" content=\"Nicolas Frey\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@nicvirtu\" \/>\n<meta name=\"twitter:site\" content=\"@nicvirtu\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Nicolas Frey\" \/>\n\t<meta name=\"twitter:label2\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/\"},\"author\":{\"name\":\"Nicolas Frey\",\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/#\\\/schema\\\/person\\\/bdca167652d883ba45818e1005505ce1\"},\"headline\":\"VMWare vSAN Series – Architecture\",\"datePublished\":\"2018-02-22T18:30:37+00:00\",\"dateModified\":\"2019-01-05T08:17:49+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/\"},\"wordCount\":552,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/02\\\/VSAN_Caching_Algo.jpg\",\"keywords\":[\"SAN\",\"Software Defined Storage\",\"VMWare\",\"vSAN\"],\"articleSection\":[\"vSAN\"],\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/\",\"url\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/\",\"name\":\"VMWare vSAN Series - Architecture - My Experience With VMware Virtualization\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/#primaryimage\"},\"thumbnailUrl\":\"http:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/02\\\/VSAN_Caching_Algo.jpg\",\"datePublished\":\"2018-02-22T18:30:37+00:00\",\"dateModified\":\"2019-01-05T08:17:49+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/#\\\/schema\\\/person\\\/bdca167652d883ba45818e1005505ce1\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/#primaryimage\",\"url\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/02\\\/VSAN_Caching_Algo.jpg\",\"contentUrl\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/wp-content\\\/uploads\\\/2018\\\/02\\\/VSAN_Caching_Algo.jpg\",\"width\":491,\"height\":264},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/vsan\\\/vmware-vsan-series-architecture\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"VMWare vSAN Series – Architecture\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/\",\"name\":\"My Experience With VMware Virtualization\",\"description\":\"Just another blog about virtualization, technology and other stuff\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/vmware.nicolas-frey.de\\\/blog\\\/#\\\/schema\\\/person\\\/bdca167652d883ba45818e1005505ce1\",\"name\":\"Nicolas Frey\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1ffd7fadaffbb5e8833858ac24afe250e8fd7d0d99a6bd3f58da665d505df69a?s=96&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1ffd7fadaffbb5e8833858ac24afe250e8fd7d0d99a6bd3f58da665d505df69a?s=96&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/1ffd7fadaffbb5e8833858ac24afe250e8fd7d0d99a6bd3f58da665d505df69a?s=96&r=g\",\"caption\":\"Nicolas Frey\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"VMWare vSAN Series - Architecture - My Experience With VMware Virtualization","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/","og_locale":"en_GB","og_type":"article","og_title":"VMWare vSAN Series - Architecture - My Experience With VMware Virtualization","og_description":"Hi everybody, in reference to my prior article\u00a0vSAN – High Available Solution\u00a0this article will explain the basic architecture and give you some insight into the technical details.","og_url":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/","og_site_name":"My Experience With VMware Virtualization","article_published_time":"2018-02-22T18:30:37+00:00","article_modified_time":"2019-01-05T08:17:49+00:00","og_image":[{"url":"http:\/\/vmware.nicolas-frey.de\/blog\/wp-content\/uploads\/2018\/02\/VSAN_Caching_Algo.jpg","type":"","width":"","height":""}],"author":"Nicolas Frey","twitter_card":"summary_large_image","twitter_creator":"@nicvirtu","twitter_site":"@nicvirtu","twitter_misc":{"Written by":"Nicolas Frey","Estimated reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/#article","isPartOf":{"@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/"},"author":{"name":"Nicolas Frey","@id":"https:\/\/vmware.nicolas-frey.de\/blog\/#\/schema\/person\/bdca167652d883ba45818e1005505ce1"},"headline":"VMWare vSAN Series – Architecture","datePublished":"2018-02-22T18:30:37+00:00","dateModified":"2019-01-05T08:17:49+00:00","mainEntityOfPage":{"@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/"},"wordCount":552,"commentCount":0,"image":{"@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/#primaryimage"},"thumbnailUrl":"http:\/\/vmware.nicolas-frey.de\/blog\/wp-content\/uploads\/2018\/02\/VSAN_Caching_Algo.jpg","keywords":["SAN","Software Defined Storage","VMWare","vSAN"],"articleSection":["vSAN"],"inLanguage":"en-GB","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/","url":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/","name":"VMWare vSAN Series - Architecture - My Experience With VMware Virtualization","isPartOf":{"@id":"https:\/\/vmware.nicolas-frey.de\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/#primaryimage"},"image":{"@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/#primaryimage"},"thumbnailUrl":"http:\/\/vmware.nicolas-frey.de\/blog\/wp-content\/uploads\/2018\/02\/VSAN_Caching_Algo.jpg","datePublished":"2018-02-22T18:30:37+00:00","dateModified":"2019-01-05T08:17:49+00:00","author":{"@id":"https:\/\/vmware.nicolas-frey.de\/blog\/#\/schema\/person\/bdca167652d883ba45818e1005505ce1"},"breadcrumb":{"@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/"]}]},{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/#primaryimage","url":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-content\/uploads\/2018\/02\/VSAN_Caching_Algo.jpg","contentUrl":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-content\/uploads\/2018\/02\/VSAN_Caching_Algo.jpg","width":491,"height":264},{"@type":"BreadcrumbList","@id":"https:\/\/vmware.nicolas-frey.de\/blog\/vsan\/vmware-vsan-series-architecture\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/vmware.nicolas-frey.de\/blog\/"},{"@type":"ListItem","position":2,"name":"VMWare vSAN Series – Architecture"}]},{"@type":"WebSite","@id":"https:\/\/vmware.nicolas-frey.de\/blog\/#website","url":"https:\/\/vmware.nicolas-frey.de\/blog\/","name":"My Experience With VMware Virtualization","description":"Just another blog about virtualization, technology and other stuff","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/vmware.nicolas-frey.de\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-GB"},{"@type":"Person","@id":"https:\/\/vmware.nicolas-frey.de\/blog\/#\/schema\/person\/bdca167652d883ba45818e1005505ce1","name":"Nicolas Frey","image":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/secure.gravatar.com\/avatar\/1ffd7fadaffbb5e8833858ac24afe250e8fd7d0d99a6bd3f58da665d505df69a?s=96&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/1ffd7fadaffbb5e8833858ac24afe250e8fd7d0d99a6bd3f58da665d505df69a?s=96&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/1ffd7fadaffbb5e8833858ac24afe250e8fd7d0d99a6bd3f58da665d505df69a?s=96&r=g","caption":"Nicolas Frey"}}]}},"_links":{"self":[{"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/posts\/320","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/comments?post=320"}],"version-history":[{"count":11,"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/posts\/320\/revisions"}],"predecessor-version":[{"id":374,"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/posts\/320\/revisions\/374"}],"wp:attachment":[{"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/media?parent=320"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/categories?post=320"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vmware.nicolas-frey.de\/blog\/wp-json\/wp\/v2\/tags?post=320"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}