Skip to content

Commit

Permalink
Fix device2yaml.rb for routeros
Browse files Browse the repository at this point in the history 
Closes #3300
device2yaml.rb
- introduces option for ssh exec mode
- the interactive output does not interpret ANSI Escape Codes anymore
- the idle timout is counted in seconds, no 0.1s timeslots anymore
- CTRL-C exits the script
- When we have a leading space, replace it with \x20 for YAML formating
routeros model
- Adds a conmandset for routeros - Thanks @systeembeheerder for this.
- Adds a YAML simulation file (CHR on GNS3)
- Adds an unit tests
- Fixes spec/model/model_helper.rb for exec mode
  • Loading branch information
@robertcheramy
robertcheramy committed Oct 25, 2024
1 parent e794481 commit d47248d
Show file tree
Hide file tree
Showing 7 changed files with 273 additions and 42 deletions.
16 changes: 11 additions & 5 deletions examples/device-simulation/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,13 @@ Run `device2yaml.rb` in the directory `/examples/device-simulation/`, the
online help tells you the options.
```
device-simulation$ ./device2yaml.rb
Usage: model-yaml.rb [user@]host [options]
-o, --output file Specify an output file instead of stdout
Missing a host to connect to...
Usage: device2yaml.rb [user@]host [options]
-c, --cmdset file Mandatory: specify the commands to be run
-t, --timout value Specify the idle timeout beween commands (default: 5 seconds)
-o, --output file Specify an output YAML-file
-t, --timeout value Specify the idle timeout beween commands (default: 5 seconds)
-e, --exec-mode Run ssh in exec mode (without tty)
-h, --help Print this help
```

Expand All @@ -68,16 +71,19 @@ this.
- When run without the output argument, `device2yaml.rb` will only print the ssh
output to the standard output. You must use `-o <model_HW_SW.yaml>` to store the
collected data in a YAML file.
- If your oxidized model uses ssh exec mode (look for `exec true` in the model),
you will have to use the option `-e` to run device2yaml in ssh exec mode.

Note that `device2yaml.rb` takes some time to run because of the idle
timeout of (default) 5 seconds between each command. You can press the "Escape"
key if you know there is no more data to come for the current command (when you
see the prompt for the next command), and the script will stop waiting and
directly process the next command.

Here is an example of how to run the script:
```
Here are two examples of how to run the script:
```shell
./device2yaml.rb OX-SW123.sample.domain -c cmdsets/aoscx -o yaml/aoscx_R8N85A-C6000-48G-CL4_PL.10.08.1010.yaml
./device2yaml.rb admin@r7 -c cmdsets/routeros -e -o yaml/routeros_CHR_7.10.1.yaml
```

### Publishing the YAML simulation file to oxidized
Expand Down
5 changes: 5 additions & 0 deletions examples/device-simulation/cmdsets/routeros
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
/system resource print
/system package update print
/system history print without-paging
/export show-sensitive
quit
110 changes: 76 additions & 34 deletions examples/device-simulation/device2yaml.rb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,36 +13,59 @@
# for its own. It works, and that should be enough ;-)

################# Methods
# runs the ssh loop to wait for ssh output, then print this output to @output,
# each line prepended with prepend
def wait_and_output(prepend = '')
# Runs cmd in the ssh session, either im exec mode or with a tty
# saves the output to @output
def ssh_exec(cmd)
puts "\n### Sending #{cmd}..."
@output&.puts " #{cmd}: |-"

if @exec_mode
@ssh_output = @ssh.exec! cmd + "\n"
else
@ses.send_data cmd + "\n"
shell_wait
end
yaml_output(' ')
end

# Wait for the ssh command to be executed, with an idle timout @idle_timeout
# Pressing CTRL-C exits the script
# Pressing ESC termiates the idle timeout
def shell_wait
@ssh_output = ''
# ssh_output gets appended by chanel.on-data (below)
# We store the current length of @ssh_output in @ssh_output_length
# if @ssh_output.length is bigger than @ssh_output_length, we got new data
@ssh_output_length = 0
# One tomeslot is about 0.1 second long.
# When we reach @idle_timeout * 10 timeslots, we can exit the loop
timeslot = 0

# Keep track of time for idle timeout
start_time = Time.now

# Loop & wait for @idle_timeout seconds after last output
# 0.1 means that the loop should run at least once per 0.1 second
@ssh.loop(0.1) do
# if @ssh_output is longer than our saved length, we got new output
if @ssh_output_length < @ssh_output.length
# reset the timer and save the new output length
timeslot = 0
start_time = Time.now
@ssh_output_length = @ssh_output.length
end
timeslot += 1

# We wait for 0.1 seconds if a key was pressed
begin
Timeout.timeout(0.1) do
# Get input // this is a blocking call
char = $stdin.getch
# If escape is pressed, exit the loop and go to next cmd
# If ctrl-c is pressed, exit the script
if char == "\u0003"
puts '### CTRL-C pressed, exiting'
cleanup
exit
end
# If escape is pressed, terminate idle timeout
if char == "\e"
timeslot += @idle_timeout * 10
puts "\n### ESC pressed, skipping idle timeout"
return false
else
# if not, send the char through ssh
@ses.send_data char
Expand All @@ -53,15 +76,19 @@ def wait_and_output(prepend = '')
end

# exit the loop when the @idle_timeout has been reached (false = exit)
timeslot < @idle_timeout * 10
Time.now - start_time < @idle_timeout
end
end

def yaml_output(prepend = '')
# Now print the collected output to @output
# as we want to prepend 'prepend' to each line, we need each_line and chomp
# chomp removes the trainling \n
@ssh_output.each_line(chomp: true) do |line|
# encode line and remove the first and the trailing double quote
line = line.dump[1..-2]
# Make sure leading spaces are coded with \0x20 or YAML won't work
line.gsub!(/^ /, '\x20')
# Make sure trailing white spaces are coded with \0x20
line.gsub!(/ $/, '\x20')
# prepend white spaces for the yaml block scalar
Expand All @@ -70,6 +97,11 @@ def wait_and_output(prepend = '')
end
end

def cleanup
(@ssh.close rescue true) unless @ssh.closed?
@output&.close
end

################# Main loop

# Define options
Expand All @@ -83,9 +115,10 @@ def wait_and_output(prepend = '')
opts.on('-o', '--output file', 'Specify an output YAML-file') do |file|
options[:output] = file
end
opts.on('-t', '--timout value', Integer, 'Specify the idle timeout beween commands (default: 5 seconds)') do |timeout|
opts.on('-t', '--timeout value', Integer, 'Specify the idle timeout beween commands (default: 5 seconds)') do |timeout|
options[:timeout] = timeout
end
opts.on('-e', '--exec-mode', 'Run ssh in exec mode (without tty)') { @exec_mode = true }
opts.on '-h', '--help', 'Print this help' do
puts opts
exit
Expand Down Expand Up @@ -132,46 +165,55 @@ def wait_and_output(prepend = '')
ssh_user,
{ timeout: 10,
append_all_supported_algorithms: true })

@ssh_output = ''

@ses = @ssh.open_channel do |ch|
ch.on_data do |_ch, data|
@ssh_output += data
# Output the data to stdout for interactive control
print data
end
ch.request_pty(term: 'vt100') do |_ch, success_pty|
raise NoShell, "Can't get PTY" unless success_pty
unless @exec_mode
@ses = @ssh.open_channel do |ch|
ch.on_data do |_ch, data|
@ssh_output += data
# Output the data to stdout for interactive control
# remove ANSI escape codes, as they can produce problems
# The code will be printed as '\e[123m' in the output
print data.gsub("\e", '\e')
end
ch.request_pty(term: 'vt100') do |_ch, success_pty|
raise NoShell, "Can't get PTY" unless success_pty

ch.send_channel_request 'shell' do |_ch, success_shell|
raise NoShell, "Can't get shell" unless success_shell
ch.send_channel_request 'shell' do |_ch, success_shell|
raise NoShell, "Can't get shell" unless success_shell
end
end
ch.on_extended_data do |_ch, _type, data|
$stderr.print "Error: #{data}\n"
end
end
ch.on_extended_data do |_ch, _type, data|
$stderr.print "Error: #{data}\n"
end
end

# get motd and fist prompt
@output&.puts '---', 'init_prompt: |-'
# YAML begin of file
@output&.puts '---'

wait_and_output(' ')
if @exec_mode
# init prompt does not exist and is empty in exec mode
@output&.puts 'init_prompt:'
else
# get motd and first prompt
@output&.puts 'init_prompt: |-'
shell_wait
yaml_output ' '
end

@output&.puts "commands:"

begin
ssh_commands.each do |cmd|
puts "\n### Sending #{cmd}..."
@output&.puts " #{cmd}: |-"
@ses.send_data cmd + "\n"
wait_and_output(' ')
ssh_exec cmd
end
rescue Errno::ECONNRESET, Net::SSH::Disconnect, IOError => e
puts "### Connection closed with message: #{e.message}"
end
(@ssh.close rescue true) unless @ssh.closed?

@output&.puts 'oxidized_output: |'
@output&.puts ' !! needs to be written by hand or copy & paste from model output'

@output&.close
cleanup
145 changes: 145 additions & 0 deletions examples/device-simulation/yaml/routeros_CHR_7.10.1.yaml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
---
init_prompt:
commands:
/system resource print: |-
\x20 uptime: 58m18s
\x20 version: 7.10.1 (stable)
\x20 build-time: Jun/27/2023 09:03:02
\x20 factory-software: 7.1
\x20 free-memory: 244.1MiB
\x20 total-memory: 320.0MiB
\x20 cpu: QEMU
\x20 cpu-count: 1
\x20 cpu-frequency: 2611MHz
\x20 cpu-load: 2%
\x20 free-hdd-space: 71.0MiB
\x20 total-hdd-space: 89.2MiB
\x20 write-sect-since-reboot: 1544
\x20 write-sect-total: 1544
\x20 architecture-name: x86_64
\x20 board-name: CHR
\x20 platform: MikroTik
/system package update print: |-
\x20 channel: stable
\x20 installed-version: 7.10.1
/system history print without-paging: |-
/export show-sensitive: |-
# 2024-10-25 06:08:35 by RouterOS 7.10.1
# software id =\x20
#
/interface ethernet
set [ find default-name=ether1 ] disable-running-check=no
set [ find default-name=ether2 ] disable-running-check=no
set [ find default-name=ether3 ] disable-running-check=no
set [ find default-name=ether4 ] disable-running-check=no
set [ find default-name=ether5 ] disable-running-check=no
set [ find default-name=ether6 ] disable-running-check=no
set [ find default-name=ether7 ] disable-running-check=no
set [ find default-name=ether8 ] disable-running-check=no
/disk
set slot1 slot=slot1 type=hardware
set slot2 slot=slot2 type=hardware
set slot3 slot=slot3 type=hardware
set slot4 slot=slot4 type=hardware
set slot5 slot=slot5 type=hardware
set slot6 slot=slot6 type=hardware
set slot7 slot=slot7 type=hardware
set slot8 slot=slot8 type=hardware
set slot9 slot=slot9 type=hardware
set slot10 slot=slot10 type=hardware
set slot11 slot=slot11 type=hardware
set slot12 slot=slot12 type=hardware
set slot13 slot=slot13 type=hardware
set slot14 slot=slot14 type=hardware
set slot15 slot=slot15 type=hardware
set slot16 slot=slot16 type=hardware
set slot17 slot=slot17 type=hardware
set slot18 slot=slot18 type=hardware
set slot19 slot=slot19 type=hardware
set slot20 slot=slot20 type=hardware
set slot21 slot=slot21 type=hardware
set slot22 slot=slot22 type=hardware
set slot23 slot=slot23 type=hardware
set slot24 slot=slot24 type=hardware
set slot25 slot=slot25 type=hardware
set slot26 slot=slot26 type=hardware
set slot27 slot=slot27 type=hardware
set slot28 slot=slot28 type=hardware
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/ip address
add address=192.168.129.7/24 interface=ether1 network=192.168.129.0
/ip dhcp-client
add interface=ether1
/ip ssh
set always-allow-password-login=yes
/system note
set show-at-login=no
quit: |-
interrupted
oxidized_output: |
# version: 7.10.1 (stable)
# factory-software: 7.1
# total-memory: 320.0MiB
# cpu: QEMU
# cpu-count: 1
# total-hdd-space: 89.2MiB
# architecture-name: x86_64
# board-name: CHR
# platform: MikroTik# installed-version: 7.10.1# software id =\x20
#
/interface ethernet
set [ find default-name=ether1 ] disable-running-check=no
set [ find default-name=ether2 ] disable-running-check=no
set [ find default-name=ether3 ] disable-running-check=no
set [ find default-name=ether4 ] disable-running-check=no
set [ find default-name=ether5 ] disable-running-check=no
set [ find default-name=ether6 ] disable-running-check=no
set [ find default-name=ether7 ] disable-running-check=no
set [ find default-name=ether8 ] disable-running-check=no
/disk
set slot1 slot=slot1 type=hardware
set slot2 slot=slot2 type=hardware
set slot3 slot=slot3 type=hardware
set slot4 slot=slot4 type=hardware
set slot5 slot=slot5 type=hardware
set slot6 slot=slot6 type=hardware
set slot7 slot=slot7 type=hardware
set slot8 slot=slot8 type=hardware
set slot9 slot=slot9 type=hardware
set slot10 slot=slot10 type=hardware
set slot11 slot=slot11 type=hardware
set slot12 slot=slot12 type=hardware
set slot13 slot=slot13 type=hardware
set slot14 slot=slot14 type=hardware
set slot15 slot=slot15 type=hardware
set slot16 slot=slot16 type=hardware
set slot17 slot=slot17 type=hardware
set slot18 slot=slot18 type=hardware
set slot19 slot=slot19 type=hardware
set slot20 slot=slot20 type=hardware
set slot21 slot=slot21 type=hardware
set slot22 slot=slot22 type=hardware
set slot23 slot=slot23 type=hardware
set slot24 slot=slot24 type=hardware
set slot25 slot=slot25 type=hardware
set slot26 slot=slot26 type=hardware
set slot27 slot=slot27 type=hardware
set slot28 slot=slot28 type=hardware
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/port
set 0 name=serial0
/ip address
add address=192.168.129.7/24 interface=ether1 network=192.168.129.0
/ip dhcp-client
add interface=ether1
/ip ssh
set always-allow-password-login=yes
/system note
set show-at-login=no
Loading

0 comments on commit d47248d

Please sign in to comment.