You may need wood to make ladders and towers to help you get over the walls and capture the town. But use only trees that you know are not fruit trees.