Transcript Network Automation

Network Automation – czyli jak można
automatyzować w sieciach.
Adam Obszyński CCIE #8557, CISSP [email protected]
Typowy interfejs systemu NMS?
Windows 95
2
© 2013 Infoblox Inc. All Rights Reserved.
NMS + zaangażowany administrator
3
© 2013 Infoblox Inc. All Rights Reserved.
Dlaczego
tak
się
dzieje?
http://0.tqn.com/d/desktoppub/1/0/t/e/3/Why.PNG
4
© 2013 Infoblox Inc. All Rights Reserved.
Bo tak :-)
Source http://www.notcot.com/archives/2006/11/world-usability.php
5
© 2013 Infoblox Inc. All Rights Reserved.
To przynajmniej działa :-)
Windows 95
6
© 2013 Infoblox Inc. All Rights Reserved.
Ale, rozmiar ma znaczenie!
http://www.polskamasens.pl/tworcza-kreska/93/Syzyf
7
© 2013 Infoblox Inc. All Rights Reserved.
Ale, rozmiar sieci ma znaczenie!
http://www.polskamasens.pl/tworcza-kreska/93/Syzyf
8
© 2013 Infoblox Inc. All Rights Reserved.
I co teraz?!
3*Z czyli ZZZ
demoty.pl
9
© 2013 Infoblox Inc. All Rights Reserved.
I co teraz?!
Zaplanuj
Zautomatyzuj
Zapomnij
demoty.pl
10
© 2013 Infoblox Inc. All Rights Reserved.
Przypadki z życia wzięte.
Network Changes
• Manual CLI, Perl scripts, and basic config back-ups
• Time intensive and requires senior engineer
Network Discovery
• Spreadsheets, periodic scans, multiple tools
• Out-of-date and incomplete data
Compliance/Standardization
• Periodic audit focus with special task force
• Adds security risk for policy violations
Access Provisioning
• Massive spike in requests and delivery expectations
• Longer SLAs - manual processes and needed expertise
11
© 2013 Infoblox Inc. All Rights Reserved.
Przypadki z życia wzięte.
Network Changes
• Manual CLI, Perl scripts, and basic config back-ups
• Time intensive and requires senior engineer
12
© 2013 Infoblox Inc. All Rights Reserved.
Ogromne centrum wystawiennicze
w słonecznej Hiszpanii.
• Czasami do 10 000 portów zmienia konfig…
• Manualny proces:
inżynier -> tester -> help desk -> dział sieci -> LAN
• Dużo pomyłek
• Ogromna pracochłonność
• Brak kontroli nad schematem konfiguracji
demoty.pl
13
© 2013 Infoblox Inc. All Rights Reserved.
Ogromne centrum wystawiennicze
w słonecznej Hiszpanii.
• Czasami do 10 000 portów zmienia konfig…
• Zautomatyzowany proces:
inżynier -> tester -> portal+API / job -> LAN
• Dużo mniej pomyłek
• Pełen „self-service”
• Dodatkowo policy & standard control
demoty.pl
14
© 2013 Infoblox Inc. All Rights Reserved.
Ogromne centrum wystawiennicze.
Z lotu ptaka.
demoty.pl
15
© 2013 Infoblox Inc. All Rights Reserved.
Przypadki z życia wzięte.
Network Discovery
• Spreadsheets, periodic scans, multiple tools
• Out-of-date and incomplete data
16
© 2013 Infoblox Inc. All Rights Reserved.
Dokumentacja sieci w koncernie
energetycznym (FR).
• Problem aktualności danych (sprzęt, soft)
• Dokumentacja sieci – wiecznie w planach
• Planowanie powolne… bo zawsze
zaczyna się od analizy tego co jest w
sieci ;-)
• Zmiany nie zawsze były optymalne
• Ręczne procesy
demoty.pl
17
© 2013 Infoblox Inc. All Rights Reserved.
Dokumentacja sieci w koncernie
energetycznym (FR) – po staremu
#1 – Sprawdź dokumentację i kiedy była
aktualizowana?
#2 – Czy rzeczywistość to czy fikcja?
#3 – Zróbmy spotkanie zespołu, może ktoś
coś zmieniał?
#4 – Wykonajmy zmianę… może będzie
dobrze.
demoty.pl
18
© 2013 Infoblox Inc. All Rights Reserved.
Dokumentacja sieci w koncernie
energetycznym (FR) - dzisiaj
#1 – Wykorzystanie aktualnych danych
(Inwentarz, Topologia, Konfigi)
#2 – Narzędzie do analizy i dyskusji na
spotkaniach – z aktualnym widokiem
#3 – Wprowadzenie zadań/skryptów i
wykonanie ich w sieci z ew. rollback.
#4 – Eksport danych + Eksport Topologii do
Visio
demoty.pl
19
© 2013 Infoblox Inc. All Rights Reserved.
Dokumentacja sieci w koncernie energetycznym
(FR) - dzisiaj
20
© 2013 Infoblox Inc. All Rights Reserved.
Dokumentacja sieci w koncernie energetycznym
(FR) - dzisiaj
21
© 2013 Infoblox Inc. All Rights Reserved.
Dokumentacja sieci w koncernie energetycznym
(FR) - dzisiaj
22
© 2013 Infoblox Inc. All Rights Reserved.
Dokumentacja sieci w koncernie energetycznym
(FR) - dzisiaj
23
© 2013 Infoblox Inc. All Rights Reserved.
Przypadki z życia wzięte.
Compliance/Standardization
• Periodic audit focus with special task force
• Adds security risk for policy violations
24
© 2013 Infoblox Inc. All Rights Reserved.
Duży BANK (USA): standaryzacja +
zgodność.
25 000 urządzeń sieciowych :-)
Prawie milion interfejsów sieciowych.
#1 – Robótki ręczne przestały się skalować
#2 – Co raz nowsze wymagania prawne
#3 – Czas (skrypt A jeszcze trwa, skrypt B
już by chciał wystartować…)
#4 – Brak centralnego światowego
repozytorium konfiguracji
#5 – Praca interaktywna (CLI^2)
demoty.pl
25
© 2013 Infoblox Inc. All Rights Reserved.
Duży BANK (USA): standaryzacja +
zgodność.
25 000 urządzeń sieciowych :-)
Prawie milion interfejsów sieciowych.
#1 – Globalne repozytorium konfigów
#2 – Backup, SLA itd..
#3 – API do integracji ( tylko jedno ;-) )
#4 – Globalne sprawdzanie policy +
raportowanie
#5 – Provision from baseline
#6 – Praca dużo mniej terminalowa
(less CLI)
demoty.pl
26
© 2013 Infoblox Inc. All Rights Reserved.
Duży BANK (USA): Przykłady…
Rule -> Policy -> Deploy == ZZZ
…
27
© 2013 Infoblox Inc. All Rights Reserved.
demoty.pl
Duży BANK (USA): Przykłady…
Rule -> Policy -> Deploy == ZZZ
…
28
© 2013 Infoblox Inc. All Rights Reserved.
demoty.pl
Duży BANK (USA): Przykłady…
…
29
© 2013 Infoblox Inc. All Rights Reserved.
demoty.pl
Duży BANK (USA): Przykłady…
Wersja CLI
use NetMRI::API::Client;
bytes.*";}
=> $s_mtu,
our $_client = new NetMRI::API::Client(
else {$cli_command_s = "show interface $s_if |
"Remote
match MTU";$cli_match = "Protocol inet, MTU: ([0Device" => $nd->{DeviceName},
UserName
=> "$http_username”,
9]+).*";}
"Remote
Password
=> "$http_password”,
print "command to push is $cli_command_s\n"; Interface"
=> $d_ifName,
URL
=> "$api_url”
$output1 = send_command($device_id,
"Remote
);
$cli_command_s);
MTU"
=> $d_mtu
our $_dis = $_client->get_broker("DisSession"); our
print "\tSource Device/interface $dev1});
$_cli = $_client->get_broker("CliConnection");
>{DeviceName}/$s_if output: $output1\n";
} else {print "\tMTUs match $s_mtu our $_issue = $_client->get_broker("IssueAdhoc"); our
$d_mtu\n";}}}
$_session_id = 0;
$d_mtu = 0;$s_mtu = 0;
sub open_session {our $_dis_response = $_dismy $dev1 = $device_id;
if ($output1 =~ m/$cli_match/){$s_mtu = $1;}; >open(job_id=> $job_id);
my $cli_command_s = "show version"; my $cli_command_d =
$_session_id = $_dis_response->{dis_session}print "\nOK, now finding far end device for
"show version"; my $cli-match;
>{SessionID};
$d_device\n”;
my $d_if; my $s_if; my $d_ifName; my $s_ifName; my
my $destdev = $broker->find_by_id($d_device);
$d_device; my $s_mtu; my $d_mtu;
print "\tGot Device $destdev->{DeviceName}\n”; END {close_session();}}
open_session();
sub close_session {our $_dis_response = $_disprint "\tNetwork device Indication is
open_connection($device_id);
>close(id=> $_session_id,);}
$destdev->{NetworkDeviceInd}\n\tManaged is $destdevmy $broker = $_client ->get_broker("Device");
>{DeviceManagedInd}\n\tCCS Collection is $destdevsub open_connection {
my $bint = $_client->get_broker("Interface");
>{DeviceCCSCollection}\n\tConfig Polling is $destdevmy $devID = shift;
$dev1 = $broker->find($dev1);
>{DeviceConfigPolling}\n”;
print "++++ Opening session to device
my $output1 ;
if (($destdev->{DeviceCCSCollection} eq "on") $devID\n";
print "\n\nCurrent neighbors of $dev1->{DeviceName}
and ($destdev->{DeviceManagedInd} eq "true") and
our $_cli_response = $_cli->open(id =>
$dev1->{DeviceIPDotted} ($dev1->{DeviceID}):\n”;
($destdev->{DeviceCCSCollection} eq "on") and
$_session_id, device_id => $devID);
my @ns = sort { $a->{ifIndex} <=> $b->{ifIndex} || $a- ($destdev->{NetworkDeviceInd} eq "true") ) {
print "DEBUG: _cli_reponse: $_cli_response\n";
>{NeighborDeviceID} <=> $b->{NeighborDeviceID} || $aif ($destdev->{DeviceVendor} eq
END {close_connection($devID);}}
>{NeighborIfIndex} <=> $b->{NeighborIfIndex} } $dev1"Cisco") {$cli_command_d = "show interface $d_if |
sub close_connection {
>get_neighbors();
include MTU”;
my $devID = shift;
foreach my $n (@ns) {
$cli_match = "MTU ([0-9]+)
bytes.*";}
our $_cli_response = $_cli->close(id =>
my $nd;
$_session_id, device_id => $devID);}
else {$cli_command_d = "show
eval {$nd = $broker->find_by_id($ninterface $d_if | match MTU";$cli_match = "Protocol
sub send_command {
>{NeighborDeviceID});};
inet, MTU: ([0-9]+).*";}
my $devID = shift;
if ($@ =~ /^H404/) { print "Could not find
print "\tcommand to push is
device $n->{NeighborDeviceID}\n";next;};
my ($command, $debug) = @_;
$cli_command_d\n”;
printf "\nOn %7s %6d %15s %16s \%s\n", ($nif($debug eq "") {$debug = 0;}
open_connection($d_device);
>{ifIndex} ? ("if" . $n->{ifIndex}) : "unknown"), $ndprint "DEBUG: Device ID is: $devID\n";
>{DeviceID}, $nd->{DeviceName}, $nd->{DeviceIPDotted},
$output1 = send_command($d_device,
$_cli_response = $_cli->send_command(id =>
($n->{NeighborIfIndex} ? ("if" . $n->{NeighborIfIndex}) $cli_command_d);
$_session_id, device_id => $devID,command => $command,
: "unknown");
debug => $debug);
my @sif = $bint->find_by_id($nprint "\tDest Device/interface $ndreturn($_cli_response->{command_response});}
>{InterfaceID});
>{DeviceName}/$d_if output: $output1\n”;
sub generate_issue {
die "\nsource Interface not found.\n\n" if
close_connection($d_device);
my ($issue_type_id, $severity, $params) = @_;
!@sif;
if ($output1 =~ m/$cli_match/)
my %baseParams = (DeviceID => $device_id,
foreach my $s (@sif) {printf "Source IF - %10s {$d_mtu = $1;}
BatchID => $batch_id, IssueTypeID => $issue_type_id,
%s\n",$s->{ifName}, $s->{ifDescr};
if ($d_mtu != $s_mtu) {
Severity => $severity);
$s_if = $s->{ifName};$s_ifName = $sprint "\tMTUs do not
my %allParams = (%baseParams, %{$params});
>{ifDescr};}
match\n”;
our $_issue_response = $_issuemy @dif = $bint->find_by_id($nmy $issue_id =
>generate_issue(%allParams);return($_issue_response>{NeighborInterfaceID});
generate_issue(
>{IssueID});}
demoty.pl
die "\nsource Interface not found.\n\n" if
"MTUmismatch”,
!@dif;
"Warning", {
foreach my $d (@dif) {printf "Dest IF
- %10s
"IP
30%s\n",$d->{ifName},
© 2013 Infoblox$d->{ifDescr};
Inc. All Rights Reserved.
Address"
=> $dev1->{DeviceIPDotted},
…
Duży BANK (USA): Przykłady…
Wersja bez CLI – Info z NetMRI
# BEGIN-SCRIPT-BLOCK
# Script-Filter: true
# Script-Login: false
# END-SCRIPT-BLOCK
use NetMRI_Easy;
my $easy = new NetMRI_Easy;
my @IFs=$easy->broker->interface->index(DeviceID=>$main::device_id);
foreach my $IF (@IFs){
my @NBs=$easy->broker->neighbor->index(InterfaceID=>$IF->InterfaceID);
foreach my $NB (@NBs){
if (defined $NB->NeighborInterfaceID) {
my $NIF=$easy->broker->interface->show(InterfaceID=>$NB->NeighborInterfaceID)>{interface};
if ($IF->ifMtu==$NIF->ifMtu){
print "Interface ".$IF->ifDescrRaw." has same MTU on neighbor\n";
}else{
print "Interface ".$IF->ifDescrRaw." (MTU:".$IF->ifMtu.") "." has different MTU
(".$NIF->ifMtu.") on neighbor\n";
my $NDevice=$easy->broker->device->show(DeviceID=>$NIF->DeviceID)->{device};
print "Neighbor Device:".$NDevice->DeviceName." Interface:".$NIF->ifDescrRaw."\n\n";
};};};};
…
31
© 2013 Infoblox Inc. All Rights Reserved.
Przypadki z życia wzięte.
Access Provisioning
• Massive spike in requests and delivery expectations
• Longer SLAs - manual processes and needed expertise
32
© 2013 Infoblox Inc. All Rights Reserved.
Bank (NL) + zespoły Managed Services
• Inny zespół od rdzenia/dystrybucji a inny
od dostępu
• Ciągłę błędy na styku dystrybucja <->
dostęp (trunking, MTU).
• Duża ilość niespodzianek w „głębokim
ukryciu”
• Ogromna pracochłonność
• Brak proaktywności
demoty.pl
33
© 2013 Infoblox Inc. All Rights Reserved.
Bank (NL) + zespoły Managed Services
• Kilka procesów (job)
• Automatyka zgodności VLAN, Trunk,
MTU (wykorzystanie tabel/list).
• Błyskawiczne wykrywanie błędów i
pomyłek
• Stała weryfikacja poprawności.
• Wykrywanie błędów przed użytkownikami
!!!
demoty.pl
34
© 2013 Infoblox Inc. All Rights Reserved.
Bank (NL) + zespoły Managed Services
Sprawdź VLAN
# BEGIN-SCRIPT-BLOCK
# Script-Filter: true
# Script-Login: false
# END-SCRIPT-BLOCK
use NetMRI_Easy;
my $easy = new NetMRI_Easy;
my @IFs=$easy->broker->interface->index(DeviceID=>$main::device_id);
my @VLANs=$easy->broker->vlan->index;
foreach my $VLAN (@VLANs){
$VLN{$VLAN->VlanIndex}{Index}=$VLAN->VlanIndex;
$VLN{$VLAN->VlanIndex}{Name}=$VLAN->VlanName;
};
foreach my $IF (@IFs){
my @IFVlans=$easy->broker->if_vlan->index(DeviceID=>$main::device_id,
IntefaceID=>$IF->InterfaceID);
foreach my $VlanId (@IFVlans){
if ($VLN{$VlanId->VlanID}{Index} and $VlanId->InterfaceID==$IF>InterfaceID) {print "Interface: ",$IF->ifDescrRaw," VLAN:",$VLN{$VlanId>VlanID}{Index},"
",$VLN{$VlanId->VlanID}{Name}, "\n";};
};
35
© 2013 Infoblox Inc. All Rights Reserved.
Przypadki z życia wzięte.
Network Changes
• Manual CLI, Perl scripts, and basic config back-ups
• Time intensive and requires senior engineer
Network Discovery
• Spreadsheets, periodic scans, multiple tools
• Out-of-date and incomplete data
Compliance/Standardization
• Periodic audit focus with special task force
• Adds security risk for policy violations
Access Provisioning
• Massive spike in requests and delivery expectations
• Longer SLAs - manual processes and needed expertise
36
© 2013 Infoblox Inc. All Rights Reserved.
Large Federal Agency
NetMRI serves as a constant monitor, and found
specific issues in the first few hours of deployment
for the agency, such as:
•
•
•
•
•
•
•
37
Configuration errors before going live
Over-temperature conditions
Redundant power-supply disconnects
Redundant link outages
Unstable or marginal WAN links and VPN connections
Spanning tree instability
Device crashes in remote offices
© 2013 Infoblox Inc. All Rights Reserved.
Infoblox Network Automation
Automated Network Discovery
Compliance & Policy Standardization
38
© 2013 Infoblox Inc. All Rights Reserved.
Change & Configuration Management
Discover
Automate
Maintain
Control
Firewall ACL & Rule Automation
Large broadband ISP
Challenge
• New naming convention
• Change 60.000 ports
• Manual 9000 manhours / several weeks
NetMRI
By creating a series of scripts in the NetMRI GUI, the
network engineering team was able to automate the
changes to interface names on 60,000 switch ports.
Script generation was extremely simple, requiring no
programming skills
Solved also similar issue: Password rotation
39
© 2013 Infoblox Inc. All Rights Reserved.
Example 1
Change:
• All switches
• Set snmp values
• Commands on Cisco:
40
•
config t
•
snmp-server community infoblox RO
•
snmp-server community netmri RW
•
end
•
write mem
© 2013 Infoblox Inc. All Rights Reserved.
The Manual Way
• Script that deals with
•
•
•
•
•
•
•
•
•
41
Login to switches
Apply the commands
Build in logging and error handling
Maintain a list of switches to run it on
Verify manually
Easy to make errors
Easy to miss errors
Time consuming
Expert user
© 2013 Infoblox Inc. All Rights Reserved.
With Infoblox Network Automation
Product does the difficult bit
Automation Logic ‘script’ on NetMRI:
Script-Filter:
$vendor eq "Cisco" and $sysdescr like /IOS/
Action:
Config SNMP
Action-Commands:
config t
snmp-server community infoblox RO
snmp-server community netmri RW
end
wr mem
42
© 2013 Infoblox Inc. All Rights Reserved.
Example 2: Like 1 But 2 Different Vendors
Without Infoblox Network Automation:
Effort doubles
With Infoblox Network
Automation:
Focus on the change itself
43
© 2013 Infoblox Inc. All Rights Reserved.
Example 3
- Not only adding the snmp values you want
- But also removing the others
- Challenge
-
You first need to see what is configured
- Automation Logic parts
-
44
Filter -> Cisco
Command to get the snmp config
Parse the output of that command and remove
© 2013 Infoblox Inc. All Rights Reserved.
Create your own – DHCP & LAN
Case #1
DHCP network/range == LAN helper address?
Case #2
LAN ip helper address == DHCP networks + range?
45
© 2013 Infoblox Inc. All Rights Reserved.
Closing the Security Lifecycle Gap
Set department policy
InfoSec
Team
•
•
•
•
•
•
•
Network
Ops
Team
Know all connected L2/L3 devices & end hosts
Automate remediation of non-compliant devices
Simplify compliance audits
Ensure security policies being followed
Reduce risk of security vulnerabilities
Integrate with SIEMs and other 3rd parties
Secure and enforce access to network
infrastructure
Deploy &
implement
Enforce &
monitor
Security
Ops Team
Discovery & Change Monitoring
Network
Ops
Team
Send for approval
Infoblox
Network
Automation
1
Alert on
violations
Discover, fingerprint, and
identify all switches, routers,
firewalls, etc., from 50
vendors
2
Backup the configurations
for all L2, L3 devices
3
Automatically check if
devices have been
changed
4
Approve change for auditing
or remediate directly
5
Deploy policies to
continuously monitor
network for compliance
Visibility & Compliance Auditing
Security
Ops Team
Send to SIEM, Network Monitoring
Infoblox
Network
Automation
1
Leverage network topology
maps to monitor for
unmanaged devices
2
Set user roles to track who
changed what and when
3
Identify network hardware
security gaps (EOL, PSIRT,
etc.)
4
Turn unused switch ports off
to reduce security profile
5
Track end hosts to determine
how network being accessed
6
Generate reports on assets,
inventories to reduce risk
Bridging the Gap
•
•
•
Set department policy
Reduced time to audit
Simplified & customizable
security policies
Out of the box compliance
InfoSec
reports
Team
• Improve agility with
automated network change
provisioning
• Inventory all network
infrastructure
• Role based access & user
auditing
Deploy &
implement
Communication
Network
and
reporting
Automation
Network
Ops
Team
• Reduce risk profile
• Continuous real-time
monitoring
• SIEM and 3rd party
integrations
Enforce &
monitor
Security
Ops Team
• Single version of truth
To działa najlepiej :-)
Windows 95
50
© 2013 Infoblox Inc. All Rights Reserved.
FIN