An AOSP based Android 4.4 KitKat ROM is already up for download for the 2013 and 2012 edition of Nexus 7. Joining them now is the Google’s last year flagship handset – the Nexus 4.
The Android 4.4 AOSP for the Nexus 4 can be downloaded from here. Keep in mind that you will need to have root access with a custom recovery pre-installed on your Nexus 4 to flash this AOSP based KitKat ROM. The accompanying Google Apps package can be downloaded from here.
Make sure to wipe the data partition since you are flashing a new version of Android on your Nexus 4.
Update: You need to have CWM recovery installed to flash this ROM. You will get an error while trying to flash the ROM from TWRP recovery.