Cisco Bug: CSCvs37013 - Prevent octeon_init from getting stuck and causing HA FTD policy deployment errors.
May 26, 2020
- Cisco Firepower Management Center
- Cisco Firepower Management Center 2500
- Cisco NGIPS Virtual Appliance
- Cisco Firepower Management Center 4500
- Cisco Firepower Management Center 4000
- Cisco Firepower Management Center 1000
- Cisco Firepower Management Center 2000
- Cisco Firepower Management Center Virtual Appliance
Known Affected Releases
Symptom: Policy deploy failed with error: "Deployment Message with id "x" cannot proceed since deployment already exists" The deployment problem is ulitmiately due to a blocked debug serial console port. In this deployment failure event, the serial console port receives a stop transmit command from the connected terminal and the serial port driver stops transmitting debug console session data to the serial port. This causes the serial port driver to report that it cannot accept any more data. The process that prints the debug session console data blocks waiting for the serial port to become free. That print process did not want to user to miss any of the console data so it blocked. Unfortunately, when the print process blocks, it was not reading from the output queue of the processor that was servicing the policy deployment. In turn, the processor responsible for the policy deployment also blocked waiting to write to the output queue that was filled up because it was not being drained. The blocked policy deploy process no longer services requests coming from the policy deployment manager and the policy deployment manager ultimately declares a policy deployment failure because the policy deployment process is non-responsive. This is a rare condition, but it did happen. The resolution is to allow the process that prints debug messages to the serial console port to be non blocking. The debug print process will buffer up data for a period of time, but when the buffers fill up, the print process will start dropping data in favor of servicing the output queue of the policy deployment process. The keeps all the processes functional and avoids policy deploy failures. Conditions: This will happen if the terminal attached to the serial console port sends a stop transmit command to the network sensor serial console port without ending the state with a start transmit command. The customers experiencing this the most were using FTD HA mode and the policy deployment events for the HA pair were printing lots of diagnostic data as they pushed the policy and communicated state between each other. The HA pair just happened to print more diagnostic data to the serial console port than a stand-alone device.
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.
Bug Details Include
- Full Description (including symptoms, conditions and workarounds)
- Known Fixed Releases
- Related Community Discussions
- Number of Related Support Cases