As posted for 14.04, here are the steps to getting a larger VNC connection to your Ubuntu 16.04 virtual machine:
Desktop Sharing
Desktop Sharing is back as an item in 16.04 desktop. Just bring up the search, type “Desktop Sharing”.
If for some reason you do not see it, type:
$ vino-preferences
Desktop Sharing Security
When your VNC client fails to connect with an error like “No matching security types” or “No security type suitable for RFB 3.3 supported” (or if you see a log line from vino-server like “Advertising security type 18”) then type:
$ gsettings set org.gnome.Vino require-encryption false
Then connect again with your VNC viewer. No restart or logout required.
Higher Resolution – a little
To get to a decent 1024×768, you can run xdiagnose:
$ sudo xdiagnose
In the dialog box, check all of the options under “Debug” : “Extra graphics debug messages”, “Display boot messages”, and “Enable automatic crash bug reporting”. Push “Apply” then “Close”.
Then, reboot
$ sudo /sbin/shutdown -r now
You can now choose 1024×768 under “Displays”
Higher Resolution – a lot
To get to higher resolutions (e.g. 1600×1200):
1) Insert the VBOX additions (from VirtualBox)
2) cd /media/$(id -nu)/VBOXADDITIONS*
3) sudo ./VBoxLinuxAdditions.run
4) sudo apt-get install virtualbox-guest-x11
and reboot again.
Now, the “Displays” settings window should allow choices from 2560×1600 to 800×600 for the VBX 0″ display.