I am not a solicitor, but will give you my perspective on this issue.
In the first instance it is ultimately the PWC who controls the level of contact with the children purely as she/he can choose to be available or not as the case maybe.
It is obviously best to sort contact between yourselves, or if this is not possible through
mediation or even using your solicitors as go betweens.
Depending on the age of the children will depend on how much input they should have in when they wish to see you. Children above the age of about 10 years old in some cases tend to talk with their feet and they themselves can be instrumental in whether to see or not to see the NRP (not always the father).
It is hoped that matters will be resolved before having to apply to the courts for contact to be sorted out. As a general rule a judge prefers to give no order as statistics show that orders are not greatly adhered to and cause further angst between parents.
There are several orders that can be granted concerning children; in your case you would probably be looking at a Defined Contact Order. This sets out conditions such as time and duration and possibly behaviours that are expected of the parents. For example, our order states that my ex-husband is not to drink alcohol 8 hrs prior to and during a contact session and that he should not swear at the children or take them into any gambling establishments.
It may stipulate that the children are to be returned on time if previously this has not happened. CAFCASS were not involved in our case but depending on reasons surrounding the application and whether contact has been witheld for a period of time etc. they maybe asked to get involved.
There are some links that I will find of things that maybe worth reading and also the Children's Legal Service that it maybe worth calling.
It is said in the Children's Act that contact with a parent or significant family member is the right of the child and what is in their best interests (not the interests of the parent).
Also it has to be remembered, a PWC can be ordered by the court to make the child available for contact with the NRP and can be penalised for not adhering to this but one cannot force a NRP to have contact with their child.
Will search out the links I mentioned later and will add them to the post.
Hope this, although not a legal input has given you some idea.
Regards, Sarah