Author Topic: The future of oldWeather  (Read 5239 times)

Bob

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1340
    • View Profile
Re: The future of oldWeather
« Reply #15 on: November 30, 2016, 11:31:57 pm »
Michael and I both have working versions of this. I'm actually using mine now for transcribing, with an automated script to draw boxes and 'type' the data in.

A spreadsheet in a smaller window over just an image of the log page to extract the weather data should work well. You wouldn't need the surrounding bit with the map and the image of the ship so the log page would be larger and easier to read, even without zooming in.

All the data will likely end up in a text file for transmitting to the science team. We're in the early stages of figuring out the content and format.

Quote
Perhaps events should go into a text file of some sort rather than a spreadsheet or just be left for the editor to do. Personally I would say extract all the info you need for the weather and leave the events for the editing process, but then I can't stop myself reading the events just in case there's something interesting.  ::)

These can all be included in the output, that way nobody misses anything.

Quote
Do we need all the separate types of events - 'animals', 'refueling' etc. tabs? Is this data being used? On the forum we're reporting aurora for space weather and volcanic activity is being passed on, so anything else that's actually useful could be handled this way.

The word is that this forum will continue.

Quote
Do we get to keep the forum or would we have to use Talk?

Excellent questions!  ;D

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 13168
    • View Profile
Re: The future of oldWeather
« Reply #16 on: December 01, 2016, 01:53:52 am »
That's a great idea, but there are some ships where weather and events are on the same page (like Jamestown 1844). Just wanted to remind everyone.

True, but the labelling of the images could help. For example:

For vessels with Events and Data on Separate pages:
  • USS_Randi_345_D.jpg
  • USS_Randi_345_E.jpg

For vessels with Events and Data on the same page:
  • HMS_Joan_345_0.jpg
  • HMS_Joan_345_1.jpg

Who is going to do the labeling?
Lots of logs are already scanned and I don't think the science team has the money to pay someone to change them and the existing references.
Even for new logs I doubt that it would be worth the extra time and effort to note what type of page was scanned.

Bob

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1340
    • View Profile
Re: The future of oldWeather
« Reply #17 on: December 01, 2016, 02:26:16 am »
As far as I know, the images will be the ones from the National Archives, similar to this one. Any further manipulation would be at the transcriber's discretion.  ;)

Who is going to do the labeling?
Lots of logs are already scanned and I don't think the science team has the money to pay someone to change them and the existing references.
Even for new logs I doubt that it would be worth the extra time and effort to note what type of page was scanned.

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 13168
    • View Profile
Re: The future of oldWeather
« Reply #18 on: December 01, 2016, 03:11:42 am »
Kevin gave me this link earlier: http://www.pmel.noaa.gov/rediscover/us-navy-logbooks
It looks like some pages are split and some are not.
https://catalog.archives.gov/id/17276893
https://catalog.archives.gov/id/7284518
https://catalog.archives.gov/id/23695631

I suspect that as Bob says they will be presented to us "as is" and that the only way to know if an image is data, events, or both is to look at it.

studentforever

  • Editor
  • Hero Member
  • *****
  • Posts: 3357
    • View Profile
Re: The future of oldWeather
« Reply #19 on: December 01, 2016, 10:42:53 am »
I've been toying with options for this. Assuming an acceptable interface is produced, I don't code so I don't know what is feasible, then how about organising it something like this.

1. Newbie contacts us - goes through tutorial

2. Newbie does trial transcription of one of a small number of previously transcribed pages - answers known so someone produces a  way of computer checking transcription and highlighting errors.

3. Option of more trial pages or choosing ship

4. Newbie then chooses ship. A brief biography of ship and example log page can be seen for each available ship. Newbie selects and is given access to a smallish block of pages which have already been transcribed once by experienced transcriber. Computer match as above to 'Ship's captain' who reviews and if appropriate advises.

5. When block finished Newbie given option of more logs on same ship or trying different ship or leaving project.

6. To start with Newbies are 2nd or 3rd transcribers - Since we do '2 out of 3 transcriptions mean agreement' I would also suggest that if we have this computer matching system we work on pages being finished with 2 transcribers agreeing. The third transcriber, if required, would be experienced and act as 'referee' on those pages which have disagreements.

7.  Each ship would have a 'Captain' who would be a sort of moderator for that ship, part of the role would be to send out blocks of pages, I think the blocks would be fairly short, maybe a month for a Newbie, and could rise to 3 or even 6 months with experience. If no response after a suitable time then transcriber would be contacted as the Naval History team will eventually do with editors. The pages could be reallocated if a satisfactory reply is not received.

Obviously this is very much open to adjustment and depends on experienced transcribers being prepared to act as captains and the really critical bit, someone producing the transcription interface and computer checking program.

I think we have to work on events. If the log will be edited then text search may produce matches, I suggest that a quick way would be keywords so that the pages containing material of potential interest could be highlighted by a simple tick box against a keyword. It would cut down the searching for researchers and when combined with edited logs would be quite powerful. It would also be quick for people mainly interested in weather, unlike transcribing and box drawing. Items which we know are of immediate interest, like volcanoes, meteors etc could be highlighted on the forum as now. But we wouold need our style of forum not talk which is basically unstructured.

Just some thoughts - when it comes to this system I think I would count myself as a Newbie.

Bob

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1340
    • View Profile
Re: The future of oldWeather
« Reply #20 on: December 01, 2016, 12:57:11 pm »
This looks like a great plan.  8)

I don't expect that there will be a formal 'interface' for this approach, at least not for a while, it would initially be run and managed through the forum.

mapurves

  • Shipherd
  • Hero Member
  • *****
  • Posts: 1836
    • View Profile
Re: The future of oldWeather
« Reply #21 on: December 01, 2016, 03:26:12 pm »
I like the idea of tick boxes on the Event Pages. I just finished some pages on Patterson where 45 crew were discharged one day and most of them re-enlisted over the next day or two. It would be much easier and quicker to be able to check a Person box, and just enter: The following men were discharged, viz.:- Tom, Dick and Harry, rather than having to draw a box and enter data for each of them.

It would be nice to have two options for when you're done with a page:
  • I'm Finished with this page; or,
  • I'm Skipping this page.

The latter would be for the case where there are Events or Data but the transcriber does not wish to do them. The page would not show as being done and the transcriber would move on to the next page. At the present time, people not wishing to transcribe the Events have to click on the "Finished" tab, which implies something which is not the case.

Event Pages should need to be done by only one transcriber.

I would like to have the option of entering a time for a location:

Name  Lat   Long  Time
Sisters Island      4 AM
          50N  122W  3 PM
etc.

Some of these ships travelling up and down the coast just note when they are abreast of one location or another at random times, and they give no noon positions.

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 13168
    • View Profile
Re: The future of oldWeather
« Reply #22 on: December 01, 2016, 03:40:24 pm »
Good ideas!

I wonder about check boxes along the lines of:
  • Untranscribed Data
  • Untranscribed Events
If you skip the page you would check 1 and 2.

mapurves

  • Shipherd
  • Hero Member
  • *****
  • Posts: 1836
    • View Profile
Re: The future of oldWeather
« Reply #23 on: December 01, 2016, 04:09:57 pm »
Good ideas!

I wonder about check boxes along the lines of:
  • Untranscribed Data
  • Untranscribed Events
If you skip the page you would check 1 and 2.

That would work.

studentforever

  • Editor
  • Hero Member
  • *****
  • Posts: 3357
    • View Profile
Re: The future of oldWeather
« Reply #24 on: December 01, 2016, 04:49:52 pm »
That sounds good but if we have mixed pages and the events have been transcribed by whatever means but we only want 1 transcription then we need a flag on the page for the 2nd person to get the page. I was thinking that anyone who gets events on their page could tick a category without necessarily transcribing anything, with or without boxes to draw. This would merely flag the presence of data concerning that topic so that researchers with access to the scans could do a very simple search to identify possible pages of interest rather than having to read the whole log. Once the events have been edited/transcribed then text searching would allow some selection within the category.

Every so often someone mentions editing the US ships. As someone who is just editing at the moment what are the plans and where would edited logs go? With so many sites going down I would hate to waste my time on a site which would 'die' before I do!

AvastMH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7409
    • View Profile
Re: The future of oldWeather
« Reply #25 on: December 01, 2016, 07:09:52 pm »
Please can we remember that the events page is often the one that contains mentions of sea ice in the US government ship log books?
Cheers,
:D

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 13168
    • View Profile
Re: The future of oldWeather
« Reply #26 on: December 01, 2016, 09:08:25 pm »
Excellent point!

studentforever

  • Editor
  • Hero Member
  • *****
  • Posts: 3357
    • View Profile
Re: The future of oldWeather
« Reply #27 on: December 01, 2016, 09:42:10 pm »
Obviously the first one in the series of tick boxes - if all else fails a tick box search will pull up the relevant pages for transcription.

The list might be
Sea ice
Animals
Volcanoes
Meteors
Kelp
Ships
Landmarks
Illness
Deaths
Crewe changes
Discipline

Do add or subtract topics, these are off the top of my head.

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 13168
    • View Profile
Re: The future of oldWeather
« Reply #28 on: December 01, 2016, 11:36:47 pm »
I don't think we care too much about meteors, but we are definitely interested in aurorae because they indicate solar activity (http://forum.oldweather.org/index.php?topic=3670.0).

I would like to know if it is worth recording comments like red patch of sea water or phosphorescence?

Bob

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1340
    • View Profile
Re: The future of oldWeather
« Reply #29 on: December 01, 2016, 11:58:27 pm »
I'd say if it interests you, record it as an Event. You never know when a Marine Biologist might find it.  ;)

I would like to know if it is worth recording comments like red patch of sea water or phosphorescence?