For a RDM and a VMFS datastore you need to create/present two separate LUNs to the ESXi hosts. What's stored on the VMFS datastore is only the RDM mapping file (this will be created at the time you add the raw LUN to a VM), which shows up with the size of the RDM LUN, but does not require disk space on the datastore (only a few bytes).
André