Cannot Delete VS: Logical Volume in Use Failure

Issue

Cannot delete a virtual server due to the error:

OnApp::Actions::Fatal Storage API Call failed: {"result"=>"FAILURE", "Error"=>"Error running lvchange: Logical volume onapp-glxi5l9ewnb3ya/mx826jsizw97zz in use.\n"}

CODE

Environment

OnApp 3.x or above

LVM Datastore

Resolution

The issue occurs due to a stuck backup iSCSI session on the backup server. Use the following steps to clear the session and remove the device:

1. Check if there are any backup iSCSI sessions open on the backup server:

[root@bk1 ~]# iscsiadm -m session | grep '172.16.4.24'
tcp: [7483] 172.16.4.24:3260,1 iqn.2014-12-14:onapp.com:backup-gzv4yhjj0vzv4u
[root@bk1 ~]#

CODE

2. Logout from the session:

[root@bk1 ~]# iscsiadm -m node -T "iqn.2014-12-14:onapp.com:backup-gzv4yhjj0vzv4u" -u
Logging out of session [sid: 7483, target: iqn.2014-12-14:onapp.com:backup-gzv4yhjj0vzv4u, portal: 172.16.4.24,3260]
Logout of [sid: 7483, target: iqn.2014-12-14:onapp.com:backup-gzv4yhjj0vzv4u, portal: 172.16.4.24,3260] successful.
[root@bk1 ~]#

CODE

3. Delete the session:

[root@bk1 ~]# iscsiadm -m node -o delete -T "iqn.2014-12-14:onapp.com:backup-gzv4yhjj0vzv4u"
[root@bk1 ~]#

CODE

4. Find the target:

[root@hv1 ~]# tgtadm --op show --mode target | grep Target
Target 1: iqn.2014-12-03:onapp.com:backup-qsgkx8eue90gyo
Target 3: iqn.2014-12-14:onapp.com:backup-gzv4yhjj0vzv4u
[root@hv1 ~]#

CODE

5. Offline the target:

[root@hv1 ~]# tgtadm --op update --mode target --tid=3 -n state -v offline
[root@hv1 ~]# tgtadm --mode target --op delete --tid=3
[root@hv1 ~]#

CODE

6. Remove the device:

[root@hv1 ~]# dmsetup remove onapp--cdw5f7xsxg8b7h-backup--gzv4yhjj0vzv4u
[root@hv1 ~]# dmsetup remove onapp--cdw5f7xsxg8b7h-backup--gzv4yhjj0vzv4u-cow
[root@hv1 ~]#

CODE