contact order

Hello,
new to the site. I am divorcing my husband as he left my 2 yr old and I 1 year ago as he was seeing another woman. This is the 3rd marriage she has successfully ruined.
I have moved 350 miles away to be near my family for support etc.For a year I did all I could to ensure my ex had contact with our son. Driving him to see him or driving him half way. I was never nasty and tried to keep things civil for my boy's sake. His father has not been civil once. Has been on 5 foreign trips not once asking to take our son with him.
He is just mean and rude he never asks after our son, what he's doing who his friends are nothing.
It culminated in him taking our son camping half way between our homes where the ex damaged his foot and rather than ring me and ask me to come and get our son he took him all the way back to his house so I could not get him back. I had to drive almost 250 miles to get him back.
The ex is living with the new piece and they both work. I am alone with my son working 12 hour shifts to make ends meet.
So I have had enough and have told the ex that he needs to see his son where we live and not drag him all the way up the country. I have never stopped contact I have always said he can see our son whenever he wants but will have to make the effort to come here to see him here for the time being. My mum even offered for him to stay at her house and she would move out for the weekend so that he could stay and not have to pay for b&b's. But no.
when he left us he said I should come and be with my family and that he would come and see our son every weekend, he has only been once. That has now all changed. He is now threatening me with court any advise?

Don't panic!

If this does go to court the aim will be to find a solution that works for everyone. However, litigation tends to leave people feeling resentful and resistant and I suggest mediation as a less confrontational way forward.

350 miles is a heck of a long way for any child, let alone a three year old, to be traveling regularly for contact. Our two used to fly about that distance as teenagers when their Dad relocated and that was bad enough.