Symptom:
High Priority Threads Backed Up On Fault Monitor Due To Thread Deadlock

Conditions:
High Priority Threads Backed Up On Fault Monitor Due To Thread Deadlock

"RdOnly-1":
  waiting to lock monitor 0x00000000004ea648 (object 0x00000007a67ee488, a com.tidalsoft.framework.rpc.RpcClient),
  which is held by "RPC Timer"
"RPC Timer":
  waiting to lock monitor 0x0000000005314ce8 (object 0x00000007a67ee470, a com.tidalsoft.scheduler.ft.FTConnection),
  which is held by "RdOnly-1"


"RdOnly-1":
                at com.tidalsoft.framework.rpc.RpcClient.execute(RpcClient.java:541)
                - waiting to lock <0x00000007a67ee488> (a com.tidalsoft.framework.rpc.RpcClient)
                at com.tidalsoft.framework.rpc.RpcClient.execute(RpcClient.java:428)
                at com.tidalsoft.framework.rpc.RpcClient.execute(RpcClient.java:404)
                at com.tidalsoft.scheduler.ft.FTConnection.broadcastFMStatus(FTConnection.java:123)
                - locked <0x00000007a67ee470> (a com.tidalsoft.scheduler.ft.FTConnection)
                at com.tidalsoft.scheduler.ft.FMNode.broadcastFMStatus(FMNode.java:176)
                - locked <0x00000007a6316210> (a java.util.Vector)
                at com.tidalsoft.scheduler.ft.BaseFTNodeMessageHandler._notifyClientStatus(BaseFTNodeMessageHandler.java:966)
                at com.tidalsoft.scheduler.ft.FMNodeMessageHandler.onFTHeartBeat(FMNodeMessageHandler.java:335)
                at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
                at java.lang.reflect.Method.invoke(Unknown Source)
                at com.tidalsoft.framework.message.MessageRouter$MessageMethod.invoke(MessageRouter.java:406)
                at com.tidalsoft.framework.message.MessageRouter._route(MessageRouter.java:205)
                at com.tidalsoft.framework.message.MessageRouter.route(MessageRouter.java:158)
                at com.tidalsoft.framework.message.BaseMessageHandlerImpl.onMessage