A good adjuster will want to meet with you in person. Why? Because they are more persuasive in person. This has been shown time and again.
As an aside, this always makes me wonder why insurance companies stop hiring field adjusters and try to do more by phone. My guess is that they save in salary for people working in an office. But, they end up paying more in settling claims.
Do not meet with the adjuster in person until your case is resolved or you have an attorney. An adjuster who is good at his or her job has been trained to meet with you in person, convince you to settle your claim, and offer you money right away. Have you ever tried to say no to someone who is offering you money right away? Yeah, it is hard to say no to that person.
Take a deep breath. You have no obligation to meet with them in person. So if you are not comfortable with it, and you shouldn't be comfortable with it, then say no.