There isn’t a way to disable Google fonts from your site – however if you do not use them on any of your pages then the Google font files will not be rendered to the page.
Only fonts which you select either on the page itself in the element options or in the Typography options for the page will be rendered to the page code – so there should be no speed impact if you just use the normal standard web ready fonts such as arial and helvetica (the ones in the non Google fonts section of any font dropdown in the system)
It’s worth noting that Google fonts are pulled directly from Google servers so there is usually minimal impact on speed of your pages if using Google fonts – however we do recommend not using more than 3 different fonts on a page to try and ensure speed is not impacted.