Preview Tool

Cisco Bug: CSCvv00859 - Multiple install snapshots in rootfs resulting in mounting file system in read-only mode

Last Modified

Sep 02, 2020

Products (1)

  • Cisco ASR 9000 Series Aggregation Services Routers

Known Affected Releases


Description (partial)

During an install operation, "EXT4-fs error" and "Buffer I/O error" are seen and the install operation fails with a 'Structure needs cleaning' error:


[126400.843219] EXT4-fs error (device dm-12): ext4_lookup:1609: inode #401229: comm instorchagt: deleted inode referenced: 414190
[126405.192299] Buffer I/O error on device dm-12, logical block 602496

instorch[277]: %INFRA-INSTALL-3-ACTION_FAILED : Packaging operation 27.1.1 failed - Internal error on 1 node. First error: 0/RP0/CPU0: [Errno 117] Structure needs cleaning:

This issue may occur on XR7 platforms running releases before 7.1.2 where this issue is fixed.

The issue can only occur if the filesystem has been exhausted of memory.  The issue is seen when the 'install-rootfs-thinpool' is full.  This can be identified by running the lvs command from the shell, or via 'run lvs' from the XR prompt, an example is in the further problem description.

The particular filesystem exhaustion tracked by this issue can only occur if the system has had several aborted 'apply by reload' install events over its uptime.  Specifically:
- These aborts must have occurred after the reloaded system reached running state with new software
- These aborts must cause the device to fail back to the old running software before the new software has been committed via <CmdBold>install commit<noCmdBold>.

In this case, a stale system snapshot is left in the 'install-rootfs-thinpool' and needs to be purged.
Bug details contain sensitive information and therefore require a account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.