Does CB Conditional really make a BIG impact?
Yes, both CB Conditional and CB Privacy have to parse every field 1 by 1. It's the only way to do field value comparison checks and privacy checks. They do have a lot of internal caching to improve on this, but you are still going to get a performance hit using them. There's no way to eliminate that, sorry. To test disable both CB Conditional and CB Privacy then see how much your load times improve.