No, that is completely fine. Components that are not part of the UI hierarchy do not exist on the client side, and are thus unreachable from the browser using any method.
Each attached component (or more generally, each “Paintable”) is assigned an internal ID for communication purposes. When a component is detached, its ID mapping is removed, so if someone were to forge a Vaadin UIDL request, the server would reject that as the ID does not match any component.
Even so, I’d advise clearly separating “cosmetics” (hiding UI elements that are not accessible) and the actual privilege enforcement that should be always done explicitly right before attempting the operation requiring elevated privileges.