I'm very sorry for the delay in responding to this.
The Home Office caseworkers do have slightly more flexibility to use discretion in regards to assessing whether unmarried partners are in a durable relationship.
For example, caseworkers are able to assess a durable relationship even when the couple have not been cohabiting for a period of 2 years. They must be satisfied that you have submitted enough evidence to show that the relationship has been existing for that length of time and is durable.
It would also help if you were able to show that you cohabited for that period of 4 years.
Due to the length of time of your relationship, a caseworker would be able to see that it is not a marriage of convenience. Being married would mean that you are a direct family member rather than an extended family member and as such you wouldn't need to apply for a family permit before coming to the UK. However, you would still need to bring evidence of the marriage and relationship with you when coming to the UK.
Let me know if you have any further questions.