By the default all project related tasks are visible to all project members.
If you want only to show the tasks where the staff member is assigned/added as follower you need to turn off Allow all staff to see all tasks related to projects
To achieve this go to Setup->Settings->Tasks and set Allow all staff to see all tasks related to projects to NO
-
If the staff member have permission for tasks VIEW or is admin then this member will be still able to see all tasks.
-