CustomerX Setup DriveScale Environment - PowerPoint Presentation

348 views
Uploaded On 2018-11-04

CustomerX Setup DriveScale Environment - PPT Presentation

DriveScale Proprietary Information 2017 Table of Contents 2 HW 2 DELL switches S4038ON 1 Management Switch 6 Server DELL PowerEdge R630 for 6 Data Nodes 1 Server DELL PowerEdge R430 for DMS ID: 713724

drivescale dms dsa adapter dms drivescale adapter dsa interface proprietary information admin https server 2017 setup root network management

Link:

Copy

Embed:

<iframe width="560" height="315" src="https://www.docslides.com/embed/713724" frameborder="0" allowfullscreen></iframe>

Download Presentation from below link

Download Presentation The PPT/PDF document "CustomerX Setup DriveScale Environment" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.

Presentation Transcript

Slide1

CustomerX Setup DriveScale Environment

DriveScale Proprietary Information ©

2017Slide2

Table of Contents

2* DELL switches S4038-ON

1* Management Switch6* Server DELL PowerEdge R630 (for 6 Data Nodes)1* Server DELL PowerEdge R430 (for DMS)2* DELL JBOD MD3060e with 60* SAS Drives each2* DriveScale DSA chassis with 4* Adapter cards eachSWNetwork config requirements: Switches: LLDP, Jumbo Frame 9K, MLAGServers: Bonding LACP, Jumbo frame 9KDSA: Bonding LACP, Jumbo frame 9KDriveScale SW: version 1.2

DriveScale Adapter Chassis (1RU)

DriveScale

2x 10GbE Interfaces per Adapter2x 12Gb 4 Lane SAS Interfaces per Adapter

Dual Redundant Power Supplies

DriveScale

Ethernet to SAS Adapters in 1 RU Chassis

With 80 Gb throughput, a single chassis can comfortably support simultaneous access to 80 drives w/ equivalent performance to Direct Attached StorageSlide4

Rack Architecture

DriveScale

Adapter

DSA

Switch 1

Switch 2

Data Node 1

MLAG



Controller 2

Controller 1

JBOD

SAS

10GE

40GE

1GE (

mngt

)

DSA

MANAGEMENT NETWORK

Management Plane Switch

DMS

(Up to 3 VM Servers for HA)

Kerberos

DHCP…Slide6

Adapter

DSA

Switch 1

Switch 2

Data Node 1

MLAG



Controller 2

Controller 1

JBOD

SAS

10GE

40GE

1GE (

mngt

)

DSA

SAS CONNECTIVITYSlide7

Adapter

DSA

Switch 1

Switch 2

Data Node 1

MLAG

SAS

10GE

40GE

1GE (

mngt

)

Adapter

DSA

DriveScale

NETWORK

HortonWorks

Hadoop NETWORK

Data Node 6

…..Slide8

Adapter

DSA 1

Switch 1

Switch 2

Data Node 1

MLAG

SAS

10GE

40GE

1GE (

mngt

)

Adapter

DSA 2

DriveScale (

iSCISI

) NETWORK

HortonWorks

Hadoop NETWORK

Data Node 6

…..

Controller 2



Controller 1

Controller 2



Controller 1

JBOD 2

JBOD 1

Final ArchitectureSlide9

DriveScale iSCSI vs Hadoop Data Network Separation

We have Six 10Gbps NICs on each server.

Four NICs are connected to a single VLAN using MLAG as the 'Hadoop' network. We have the DNS routing for the FQDN of each machine set so it resolved them to the IP address on that VLAN. That is the one that Hadoop,

ssh, etc. would use to contact the server.The other two NICs are connected to a separate VLAN using MLAG as the 'DriveScale' network that is part of the subnet and DriveScale bandwidth domain with the adapters. To achieve the separation that CustomerX wants, that VLAN should not have a default route set on the servers, nor any device willing to route traffic.All NICSs are connected through a single pair of switches, with one (or two) link (s) of each VLAN/MLAG pair going to each switch.

HTTPS (80)

SSH (22)

Drivescale

Central

HTTPS (443)

HTTPS

HTTPS/SSH

HTTPS (8443)

Zookeeper (2888, 3888 TCP)

MongoDB (27017)

Zookeeper

(2281 TCP)

Zookeeper

(2281 TCP)

iSCSI (wide port range)

HTTP (38202)

HTTPS

HTTPS (8444,443)

Communication Flow – Diagram Slide11

Communication Flow – TCP ports for Firewall Policy

User -> DMS - Management interface access (HTTPS , HTTP redirect to HTTPS on 80)

if using

haproxy for HA, then HTTPS of individual server (not haproxy) is on 8443User -> Adapter - Network configuration - HTTPSAdapter -> DMS - software updates (HTTPS on 8444)DMS -> DriveScale Central - software updates (HTTPS)DMS -> DriveScale Central - statistics / logs upload (HTTPS)DMS -> DMS -

haproxy

health check (HTTPS on 8443)

DMS -> DMS - VRRP for HA setup

DMS -> DMS - Zookeeper replication (2888, 3888 TCP)

DMS -> DMS - MongoDB replication + access (27017 TCP)

Server/Adapter/DMS -> DMS - Zookeeper access (2181 TCP)

Server -> Adapter - iSCSI (wide port range, one iSCSI portal per drive used)

Server -> Adapter, HTTP on alternate port 38202 for balancing (Live Data load Monitoring Reported from Adapter).

Server ->

DriveScale Central - software install (HTTPS)Server/Adapter -> DMS Log Report (HTTPS)

DriveScale Setup - Prerequisites

You must complete the racking and cabling of the Servers, DSA, JBOD’s and switches according to the details of the network requirements.

You must obtain the credentials of your

DriveScale Central account which is shipped with hardware.You must decide whether to set up your DMS as one standalone server or as a high-availability cluster with three servers. When three are used, the Management Domain can survive the failure of any one of the DMS machines. The DMS servers should be configured with minimum HW Requirements (depends on targeted number of DSAs/Nodes/Drives to be supported). DHCP server in the 10G/1G network (unless using static IP assignment)Access to the DHCP administrator/server to get the IP address(

) of the DSA(s) based on the MAC address(

) of the DSA(s).

The network address(

) of your DMS server(s).

The network address(

es) of your DSA(s).The network addresses of your compute servers.

Go to https://central.drivescale.com/

DriveScale. A checklist of the tasks that need to be accomplished appears on the main DSC page.

DriveScale

Central – login

1. DriveScale

Central (

Cont

…) – create your

DriveScale Management Domain14Go to the Domains link in the left navigation panel and click on Create Domain.

Fill the name, FQDN name and any notes for the domain. Click on

Create

1. DriveScale

Central (

Cont

…) – download packages

15Go to the Downloads link in the left navigation panel and download the config-Training,ds-dms-keys-xxx.rpm and ds-repo-xxx.rpm files on your local machine.

2. DMS Setup – HW/SW Requirements

Supported OS: Linux RHE or Centos 6/7, Ubuntu 14.x.

The below options will cover DMS resource for managing up to 125 Nodes - 125 DSA adapter cards - 1250 Drives. Once you get closed to this threshold, we’ll ask you to add more resource to the config (CORE/RAM/DISK SPACE).

Option 1 (recommended) : 3x Servers (HA), each with the following config:2 cores8 GB RAMA local SSD for OS + ZooKeeper transaction logs (size >= 100 GB).A local SSD for the ZooKeeper snapshots (size >= 100 GB)1

GbE

NIC

Option 2: Single Server with 2x SSDs (in RAID1):

2 cores

8 GB RAM

2x local SSDs for OS +

ZooKeeper

transaction logs +

ZooKeeper snapshots (size >= 200 GB).1 GbE NIC

2. DMS Setup (Cont

…) –

Overview

Copy RPMs and the “DriveScale Domain” Config file received from Drivescale Central to the DMS host.Install the RPMs on DMSCopy config file to drivescale directory on DMS

Start the

Drivescale

Agent Service

Verify the DMS is running and GUI is accessible from your host

2. DMS Setup (Cont

…

) – install packages

Copy RPMs and config file received from Drivescale Central to DMS from your localhost (via scp or sftp) [root@localhost]# scp ds-* config-*

root@dms_ip

tmp

b) SSH to your DMS, install the RPMs and ds-

dms

service

[root@dms]# cd /tmp [root@dms]# rpm -

ivh ds-repo-* ds-dms* [root@dms]# yum install ds-

dms –y Note: It is recommended to do a YUM UPDATE to update your current Linux packages.

c) Copy the config file you received from Drivescale Central to DMS drivescale directory:

[root@dms]# cd /tmp[

root@dms]# cp config-* /etc/drivescale/

conf && chmod 660 etc/drivescale/conf

2. DMS Setup (Cont

…

) – prepare Zookeeper

The DriveScale Management Server uses the ZooKeeper package to manage the configuration of elements of the system. If there will be more than about 500 servers and DriveScale Adapters (DSA) in use in the data center, the number of open file descriptors available to ZooKeeper may need to be increased, so that it can maintain a connection to the agents on each server and DSA.The following files needs to updated in CentOS/RHEL:

- /

etc

/security/

limits.conf

root -

nofile

32768

- /etc/

sysctl.conf fs.file-max = 32768

2. DMS Setup (Cont

…

) – start

DriveScale Service

20 d) Start the Drivescale Agent ServiceNote: In below example, we turn off Firewall and SELINUX on the DMS installed on a Linux RHE7 or Centos7 host. If you do want to keep the Firewall on, make sure the required TCP ports are open as described in Communication Flow slide (10) .# turn off firewall

[

root@dms

systemctl

stop

firewalld

[

root@dms]#

chkconfig firewalld off#disable SE linux[

root@dms]# sed -i 's/(^SELINUX=).*/SELINUX=disabled/' /etc

/sysconfig/selinux[root@dms]#

sed -i 's/(^SELINUX=).*/SELINUX=disabled/' /etc/selinux

[root@dms]# reboot …

# Verify SE linux status[root@dms]#

sestatus# Start DriveScale service

[root@dms]# systemctl start ds-dms

#Run DMS setup script to initiate DMS WEB UI local authentication [root@dms]# /

opt/drivescale/bin/setup-mode

2. DMS Setup (Cont

…

) – connect to DMS Web UI

e) Open a browser and connect to DMS on port 443: https://dms_ip f) Setup your DMS Web UI credentialsg) Reconnect to your DMS URL and login

with your credentials.Slide22

3. DSA Setup – overview

Each

DriveScale

Adapter (DSA) has 3 physical Ethernet interfaces and 3 Logical interfaces:One 1Gb Ethernet physical Interface:Used for management purpose onlyTwo IP interfaces (and MACs) are mapped to this 1GE interface for the following 2 logical interfaces:One logical SUP Interface — Supervisor interface for the adapter. DMS communicates with this interface for configuring the adapter.

One logical management interface — Management interface for the adapter.

This interface can be used by the adapter for

sending

updates to the DMS (optionally to segregate Control Plan Traffic between DSA and DMS)

Two 10Gb Ethernet physical interfaces

These are the data interfaces.

They are used for connecting compute resources to the JBODs

They can be used by the adapter for sending updates to the DMS (optionally share Control Plan and Data Traffic between DSA and DMS)

By default, the Two 10Gb interfaces are bonded (LACP based)

A single IP interface is mapped to this Bonded interface By default, all IP interfaces are requesting IP assignment via DHCP

DriveScale Adapter (DSA) Network interfaces can be configured from DMS using the DSA Utility tool described in the next slide.

Complete commands and usage of DSA utility tool is described in the administration guide.

3. DSA Setup (Cont

…

) – Discover IP via DHCP

Adapter

DSA

Chassis

Management Plane Switch

DHCP Server

Sup MAC-Adapter1

Sup MAC-Adapter2

Sup MAC-Adapter4

Sup MAC-Adapter3

1. Save MAC@ of each Adapter Supervisor interface (see printed on chassis or/and packaging box)

2. Connect a DHCP server to Management and provide IP (and FQDN) assignment in same subnet as DMS (or routable to DMS)

3. Save IP/Mac mapping of each Adapter Supervisor interface

DMS

MAC-DMS

FQDN: dms.CustomerX.com

Static IP: 10.0.0.10/24

GW: 10.0.0.1

DHCP Request

DHCP OfferSlide24

3. DSA Setup (Cont

…

) – configure the DSA adapter

Log into one of the DMS machines using ssh.Set up the DMD configuration for each DSA using the same config file as was used in the DMS. This is done via the /opt/drivescale/bin/dsa

script command available in Linux shell of the DMS host:

The script runs always from DMS towards the Supervisor IP of the DSA adapter

The default DSA adapter Supervisor

username/password

admin/admin (

this can be changed via configuration if needed).

Run the command listed below to check the current configuration and settings of each DSA adapter:

/opt/drivescale/bin/dsa --username admin --password admin --adapter

service showconfPush the config from the DMS to the DSA adapter:

/opt/drivescale/bin/dsa --username admin --password admin --adapter service config --file /

etc/drivescale/conf Restart the DSA adapter

/opt/drivescale/bin/dsa --username admin --password admin --adapter service restart

3. DSA Setup (Cont

…

) – set the DSA adapter with static IPs

SSH to the DMS hostConfigure the Supervisor interface with static IP/gateway… etc: /opt/drivescale/bin/dsa --username admin --password admin --adapter

net modify --interface sup --static 10.0.0.11 --netmask 255.255.255.0 --gateway 10.0.0.1 --

fqdn

a1m.dsa1.CustomerX.com

Verify the Supervisor interface Static IP’s address:

/opt/

drivescale

/bin/

dsa --username admin --password admin --adapter net show --interface supConfigure the Management interface with static IP/gateway…

etc: /opt/drivescale/bin/dsa --username admin --password admin --adapter

net modify --interface mgmt --static 10.0.0.12 --netmask 255.255.255.0 --gateway 10.0.0.1 --fqdn a1m.dsa1.CustomerX.com

Verify the Management interface Static IP’s address: /opt/drivescale/bin/dsa --username admin --password admin --adapter

net show --interface mgmtConfigure the 10Gbps Bond interface with static IP address:

/opt/drivescale/bin/dsa --username admin --password admin --adapter

net modify --interface 10gBond --static 192.168.1.11 --netmask 255.255.255.0 --gateway 192.168.1.1Verify the 10Gbps Bond interface Static IP’s address: /opt/drivescale

/bin/dsa --username admin --password admin --adapter net show --interface 10gBond

Set up DSAs

11. Confirm the DSAs are setup when the

Drivescale

Management Serfver (DMS) sees them in physical inventory.

Note: Once the DSA is found, you will like also find several connected device, discovered by LLDP, such as switches, JBODs, and drives.

4. Troubleshooting – get logs

SSH to the DMS host

DSA logs: run the command against the Supervisor interface to collect logs package of the DSA:

/opt/drivescale/bin/dsa --username admin --password admin --adapter logs

It should download the logs in current directory (where you run the command on DMS) as xxxx-logs.tgz

DMS logs are in