Author Topic: Marking Time  (Read 6061 times)

philip.brohan

  • Old Weather Team
  • Full Member
  • *****
  • Posts: 198
    • View Profile
Marking Time
« on: December 14, 2010, 04:14:01 pm »
Quite a few people have asked why we don't have to input the time of each weather observation. It's a sensible question, and we do need the observation times, particularly for tracking fast-changing weather events like moving fronts. But one of the clever features of the design of the oldweather website is that we don't have to enter the times - they are automatically collected through the process of entering the weather data.

To do this we take advantage of a symmetry between space and time in the logs (scientists love symmetries). The top of each log page corresponds to the beginning of the day, and the bottom of the page corresponds to the end of the day. So the further down the page an entry is, the later in the day it was taken.  We record the position of the push-pin for each entry digitised from the page and, from that push-pin position, we can find the time associated with that entry.

The image shows the positions of all the weather observations entered from the logs of HMS Bacchante (thanks captain richbr15, lieutenants dazedandconfused and davemcg, and all the crew).
As with most books, there are two sorts of pages: left-hand (red dots) and right-hand (blue dots). They have different margins in our images, so they don't line up precisely horizontally, but their vertical position is the same, and that's what gives us the time.

The Bacchante recorded the weather at the end of each watch: so at 4, 8 and 12 a.m., and the same times in the afternoon. They also recorded it at the end of the first dog watch (6 p.m.) - so we should expect to see three equally-spaced groups of points in the top half of the figure (the morning), and four, more irregularly spaced, groups in the bottom half (the afternoon). This is exactly what we see, and it's clear that for the vast majority of the observations, we can easily say which watch they are associated with, and so when they were taken.

There are a few observations that are not quite so easy - we can see some smaller clusters of observations above and to the left of the main clusters; but again, it's easy to see which watches there observations correspond to. There are also a few observations in irregular positions - lost in time and space - but these are only a tiny fraction of the total.

So it's going to be easy to find the time of observation in the usual case where the observations are 2 or 4 hours apart. For the diligent few log-keepers who recorded observations every hour or even more frequently, we will have to be a bit cleverer; and use the differences between the observed weather values, as well as the position on the page, to group the observations into clusters and assign them to times.

All this, of course, relies on having accurate positions on the page for each observation, which means lining up the entry box with the observation text carefully each time when entering the data. So far, we've done well at this (as the figure shows); I've come to expect no less from oldweather, but still thanks - it makes my job easier.

Philip

cyzaki

  • Hero Member
  • *****
  • Posts: 855
    • View Profile
Re: Marking Time
« Reply #1 on: December 14, 2010, 05:54:25 pm »
Brilliant - I love all the sciency feedback! Keep it coming  ;D

pliget

  • Full Member
  • ***
  • Posts: 121
    • View Profile
Re: Marking Time
« Reply #2 on: December 14, 2010, 07:47:47 pm »
Thank you. I like totally agree with cyaki :).

The more we understand about how the data is used the better we can be in producing it I think.

I presume however, that allowance is made for the push pin position vs the data. My pin for 2400 for instance, will probably be at about the 2300 hrs level as it them highlights the 2400 hrs data in the box below it. That seems to be borne out by your chart which shows the last set of readings clustered around 0860 when the data is more at 0950. Complete rubbish as normal :). The pin is positioned accurately.

« Last Edit: December 15, 2010, 06:28:46 pm by pliget »

DJ_59

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 836
    • View Profile
Re: Marking Time
« Reply #3 on: December 15, 2010, 01:13:37 am »

Thanks, Philip!

Janet Jaguar

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10166
  • Smell the sea, feel the sky, & fly into the mystic
    • View Profile
Re: Marking Time
« Reply #4 on: December 15, 2010, 02:03:35 am »
Thanks from me, too.  The graphic makes it wonderfully clear.

tastiger

  • Shipherd
  • Hero Member
  • *****
  • Posts: 792
    • View Profile
Re: Marking Time
« Reply #5 on: June 05, 2011, 05:08:37 am »
Just wondering, since event times are based on weather entries, what is going to be done about ships like Trent and Raven II?

Janet Jaguar

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10166
  • Smell the sea, feel the sky, & fly into the mystic
    • View Profile
Re: Marking Time
« Reply #6 on: June 06, 2011, 01:50:10 am »
Just wondering, since event times are based on weather entries, what is going to be done about ships like Trent and Raven II?

The team and their analysts are fully informed of which ships have non-standard logs, whether it is because the printed form is different or the log-keeper is a stubborn fisherman who is more than willing to serve his country with his ship - but has no intention of having to change his paperwork at the same time!  Which probably means some poor soul is actually going to have to look at every page and reassign the time of day to every transcribed record.  This analysing team is also why we have a single thread reporting all the bad scans.

Anyone want to place any bets that more of such logs are also going to show up in the second phase of the project? ;D

tastiger

  • Shipherd
  • Hero Member
  • *****
  • Posts: 792
    • View Profile
Re: Marking Time
« Reply #7 on: June 06, 2011, 02:42:02 am »
Just wondering, since event times are based on weather entries, what is going to be done about ships like Trent and Raven II?

The team and their analysts are fully informed of which ships have non-standard logs, whether it is because the printed form is different or the log-keeper is a stubborn fisherman who is more than willing to serve his country with his ship - but has no intention of having to change his paperwork at the same time!  Which probably means some poor soul is actually going to have to look at every page and reassign the time of day to every transcribed record.  This analysing team is also why we have a single thread reporting all the bad scans.

Anyone want to place any bets that more of such logs are also going to show up in the second phase of the project? ;D
Yeah, I'll bet there'll be quite a few!  ;D

And another thing... What about the pages where only some of the weather entries are in (like in this case http://oldweather.s3.amazonaws.com/ADM%2053-63334/ADM%2053-63334-012_0.jpg and this one http://oldweather.s3.amazonaws.com/ADM%2053-63334/ADM%2053-63334-012_0.jpg) and ones with information in the events but the log keeper forgot to put in the weather readings (another Torch page, but I can't find it right now)?
« Last Edit: June 06, 2011, 02:46:23 am by tastiger »

Janet Jaguar

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10166
  • Smell the sea, feel the sky, & fly into the mystic
    • View Profile
Re: Marking Time
« Reply #8 on: June 06, 2011, 03:40:39 am »
Quote
And another thing... What about the pages where only some of the weather entries are in (like in this case http://oldweather.s3.amazonaws.com/ADM%2053-63334/ADM%2053-63334-012_0.jpg ...

Those are much easier, tastiger.  Every weather record that exists is in the correct place for that time of day.  And every weather record that was forgotten is blank in the correct place for that time of day.  No problems at all, save maybe a frustrated desire to find a TARDIS and go back there to tell the log-keeper how important this is to the 21st century.

For what it's worth, there is an option in the wind direction drop-downs, type "b" and it will offer you "blank".  Not that you have to do that to let the emptiness say the same thing.  Remember, they will have 3 transcribers dittoing all those blank readings. ;)

Thursday Next

  • Editor
  • Hero Member
  • *****
  • Posts: 2425
    • View Profile
Re: Marking Time
« Reply #9 on: June 06, 2011, 02:22:52 pm »
Just wondering, since event times are based on weather entries, what is going to be done about ships like Trent and Raven II?

I don't know about Trent, but I've done a few pages on Raven.  Although the layout of the pages is different from the standard logbook, all her weather entries are precisely timed, so you can still get the timing from the position on the page.  The conversion would be different obviously.

Janet Jaguar

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 10166
  • Smell the sea, feel the sky, & fly into the mystic
    • View Profile
Re: Marking Time
« Reply #10 on: June 06, 2011, 04:41:47 pm »
I did a bit on one of the trawlers - just to see what was so different that everyone was putting it on the forum.  Then I did a quick look-see on everything else on the non-standard list.  They were all private vessels that had been drafted into the navy for the war, and those with good log books provided by their normal bosses apparently weren't asked to change anything.  Which makes sense, if you are keeping her own crew who are used to it.  The analysts will just have to set a new template for identifying and timing the logs, not a problem

But the armed trawlers requisitioned, Tenby Castle and HMS Saxon - both now finished - were clearly fishing boats still being run by their owners, who didn't do log-keeping in the normal way at all.  So the RN gave them standard blank books to fill in.  Both of them made exactly the same decisions:
  • Forget the time thing, they really only needed half a page per day, so that's all we'll use.
  • And who needs to record all those itty bitty weather numbers, when all any wise fisherman needs to do is stick his head outside and look?  I guess I'll describe it, and maybe once or twice a day look at the barometer.

They sound like very interesting people to know. ;D

Bunting Tosser

  • Hero Member
  • *****
  • Posts: 2697
    • View Profile
Re: Marking Time
« Reply #11 on: June 06, 2011, 04:47:20 pm »
And who needs to record all those itty bitty weather numbers, when all any wise fisherman needs to do is stick his head outside and look?  I guess I'll describe it, and maybe once or twice a day look at the barometer.

"You don't need a weatherman to know which way the wind blows."

ElisabethB

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 539
    • View Profile
Re: Marking Time
« Reply #12 on: June 06, 2011, 09:20:13 pm »
"You don't need a weatherman to know which way the wind blows."

So true, my friend, so true !  ;) ;D