Activity feed isn't user specific; it's domain specific, which is why it's under Site scope. You can read about how it's used more specifically below.
developers.facebook.com/docs/reference/plugins/activity/
Facebook doesn't offer what you're wanting as a Social Plugin so naturally CB Connect can't provide what does not exist. It's possible through Facebook API, but if you did it on a scale of user by user you'd hit API limits daily and then they'd ban your application indefinitely.