The way you configure the ESXi initiator for VN2VN mode is as follows (command line only)
Example commands
===============
1. Create Vswitch
esxcfg-vswitch -a vs2;
2. Setup VMNIC that you wanted to use for VN2VN connection as uplink to vSwitch
esxcfg-vswitch -L vmnic2 vs2;
3. Enable VN2VN Mode on the adapter first
esxcfg-fcoe -V vmnic2;
NOTE: By default FCOE adapter is setup in the FABRIC mode only if you do not specify the VN2VN mode intention.
4. Add VLAN and Start Discovery
esxcfg-fcoe -v 569 -d vmnic2;
This should solve your problem.