talking to myself seems like the drama is over.
Brocade adapters with updated boot code v3.2.6.0 are getting through installation phase of ESX v6 and after manual driver installation (esxcli software vib install -v) they show up as networking adapters. While I didn't perform extensive testing, at first glance and after some gigabytes transferred left and right it seems to be OK. Because they are on VMware HCL, hopefully there will be no more headaches - I've had enough.
I'm using driver v3.2.5 because REAL v3.2.6 is nowhere to be found as described above (qlogic webpage takes you to vmware.com download and there is v3.2.4 only, despite name saying v3.2.6). I'm pretty sure this will be corrected shortly.
Nobody can explain me why no-name whitebox works OK with these adapters (UEFI BIOS of course) ; and why the same DL380 Gen9 servers are OK with ESX5.1 and 5.5.
To be honest, I didn't suspect Brocade 1020 firmware / boot code to be the culprit.
Took me three days. And I still have to pull out 14 adapters from servers and update them in external enclosure with Windows 2012. Duh.