Guest

Preview Tool

Cisco Bug: CSCvt92190 - Stuck VSH sessions attempting to cat(1) mvsh_cmd to mvsh_op resulted in login failures, # fopen

Last Modified

Sep 11, 2020

Products (1)

  • Cisco Nexus 9000 Series Switches

Known Affected Releases

7.0(3)I1(1) 7.0(3)I7(7)

Description (partial)

Symptom:
When trying to log in to a Nexus switch over SSH or Console we see the following error:

Problem Details: timer lib: could not get cpu_hz
timer lib: could not get cpu_hz
ERROR: sse_fd=-1 err=23 sse_lib failed to open the /dev/klm_sse
ERROR: sse_fd=-1 err=23 sse_lib failed to open the /dev/klm_sse
Couldn't open /dev/null: Too many open files in system

Conditions:
It appears that a large number of "vsh" processes are open and this is causing new VSH login sessions to fail

<pre>
# NX-OS 7.0(3)i1(x)
[ snip ]
root     10361  0.0  0.0 308468  8840 ?        Ss    2017   0:00  |   \_ sshd: sa-cisco-prime@pts/24                                                                
2012     10368  0.0  0.0      0     0 ?        Zs    2017   0:00  |   |   \_ [vsh] <defunct>                                                                        
root      5475  0.0  0.0 308468  8820 ?        Ss    2017   0:00  |   \_ sshd: sa-cisco-prime@pts/25                                                                
2012      5482  0.0  0.0      0     0 ?        Zs    2017   0:00  |   |   \_ [vsh] <defunct>                 
[ snip ]
2008     11221  0.0  0.1 370300 17216 ?        Ss    2017   0:00 /isan/bin/vsh
2008     27959  0.0  0.1 370300 17208 ?        Ss    2017   0:00 /isan/bin/vsh
2008      2166  0.0  0.1 370296 17208 ?        Ss    2017   0:00 /isan/bin/vsh
2008     28723  0.0  0.1 370304 17208 ?        Ss    2017   0:00 /isan/bin/vsh
2008      9178  0.0  0.1 370304 17208 ?        Ss    2017   0:00 /isan/bin/vsh

# NX-OS 7.0(3)i5(x)
root     31534  0.0  0.0   4996  1068 ?        S     2019   0:00 sh -c cat /tmp/mvsh_cmd.7256065.0.1 | /isan/bin/vsh -N -p admin -o 64 >> /tmp/mvsh_op.7256065.0.1 2>&1
admin    31551  0.0  0.2 561476 39532 ?        S     2019   0:00  \_ /isan/bin/vsh.bin -p admin -o 64
root     31539  0.0  0.0   4996  1068 ?        S     2019   0:00 sh -c cat /tmp/mvsh_cmd.7256066.0.1 | /isan/bin/vsh -N -p admin -o 64 >> /tmp/mvsh_op.7256066.0.1 2>&1
admin    31553  0.0  0.2 561476 39528 ?        S     2019   0:00  \_ /isan/bin/vsh.bin -p admin -o 64

# NX-OS 7.0(3)i7(x)
root     31534  0.0  0.0   4996  1068 ?        S     2019   0:00 sh -c cat /tmp/mvsh_cmd.7256065.0.1 | /isan/bin/vsh_perm -p admin -o 64 >> /tmp/mvsh_op.7256065.0.1 2>&1
admin    31551  0.0  0.2 561476 39532 ?        S     2019   0:00  \_ /isan/bin/vsh.bin -p admin -o 64
root     31539  0.0  0.0   4996  1068 ?        S     2019   0:00 sh -c cat /tmp/mvsh_cmd.7256066.0.1 | /isan/bin/vsh_perm -p admin -o 64 >> /tmp/mvsh_op.7256066.0.1 2>&1
admin    31553  0.0  0.2 561476 39528 ?        S     2019   0:00  \_ /isan/bin/vsh.bin -p admin -o 64
</pre>
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.