The Task field in the expense sheet cannot be disabled for specific users using permissions. It can be enabled or disabled globally for all users. However, individual users can disable it from their expense sheets Settings, if they do not want to use the field or have no requirement for the field.