After all the calculations, especially considering the glass in front of sensor, I think I am abandoning this project. I would probably have to machine at least 3 mm off to get close to infinity focus, leaving a too thin of a lip. Also considering the Fuji crop factor,I would not be getting the moderate(underwater)wide angle option the 15mm presents. I will just use for macro with the 80mm and maybe see how the 15 does with closeups.
Thanks for everyones input.