A very basic check list when reviewing papers.
Kees Goossens

Some things to check for:
- do title, abstract, introduction, and conclusions match?
- are the claims in the abstract & introduction justified, i.e. are they proven?
- is the problem defined well?
- are the solution (and its concepts) defined well?
- are the limitations & constraints given?
- is the paper well-structured in terms of sections, layout, figures, English?
- is it understandable?
- is there a related work section?  is it correct & fair to the other papers?
- are all related papers that you know cited?  'selective' citing is not allowed
- does it describe the relation of those works to the paper (and not just a description of what the related work does)?
- are the experiments defined in sufficient detail (could you reproduce them based on the paper & the refs)?
- are the experiments presented well (graphs, etc. understandable)?
- are the conclusions that are drawn from the experiments justified?
- what is the novelty of the paper?
- is it a cool idea? give kudos bonus points for this
- has it been done a million times before? does it add anything to existing papers? if not, reject
- regarding expertise level: if you're doing your thesis on the topic, you're an expert

You can write the review as a list of bullets (my style, less nice),
or as running text (most other people's style, recommended, nicer).
In the comments to the TPC write a max 5 bullet summary.