Author Topic: oldWeather teleconferences  (Read 1663 times)

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 12985
    • View Profile
oldWeather teleconferences
« on: March 29, 2016, 07:50:26 pm »
Oldweather teleconference 17 March 2016

This was a meeting of members of the OW science team and the Zooniverse team.
I was there, for the first time, to represent the transcribers and moderators. To tell the truth, I was so nervous beforehand that I had an upset stomach, but I rather enjoyed the meeting.



OW1 and OW2 (WW1 Royal Navy) editing continues, slowly but steadily. Well over 200 fully edited logs have been published on naval-history.net. The observations are in use.

OW3 (Classic) transcribing continues. Many observations are already in use, but additional processing is being done to convert the data to the standard International Maritime Meteorological Archive format and for quality control. The reanalysis team have used all of our observations to date, and they are in the process of developing a new, improved twentieth-century reanalysis.

OW4 (Whaling) has been very popular. However, the downside is that we are overwhelming the system with data - especially during the marking phase. The moderators are maintaining a list of known issues. We are hoping to add some short logs in order to avoid the most serious problems. We are currently looking for someone to work on this (any ReactJS and Ruby on Rails experts among our US members?).

OW5 (Panoptes) has been less popular than OW4. I suggested that the random page allocation was a major factor, but there are other problems too. The moderators are maintaining a list of known issues. We hope that work on improvements to OW5 will start in about a month.

OW6 (Optical Character Recognition) is in development. It is designed to combine OCR technology and citizen science in order to make transcribing logs less laborious. I don't think that we are going to lack for work in the foreseeable future! First, we will be needed to provide some training/validation examples. In fact, we should be getting some typed logs sometime soon. Second, OCR will be primarily used for typed logs. It was mentioned that the US Post Office uses OCR, but there are significant differences between an address and a log page!

Talk is used by OW4 and OW5, and they share the same Talk site (https://www.zooniverse.org/projects/zooniverse/old-weather/talk/). This has caused some confusion. The possibility of each project having its own Talk was raised, but the disadvantages are that this might split the community and that things like Geographical Help and the Reference Desk are used by both projects. I was told that there will be further development on Talk soon and was asked for a list of suggested enhancements. The moderators have already created the list and sent it off! ;)

Next year there will be a field campaign in the Greenland/Iceland/Norwegian (GIN) seas. We might partner with them and do a parallel 'office' campaign recovering similar data from the Greenland Patrol logs in the WW2 period (Aivik).



I reminded everyone that there are plenty of volunteers who would be willing to help with the testing ;D

mapurves

  • Shipherd
  • Hero Member
  • *****
  • Posts: 1772
    • View Profile
Re: oldWeather teleconferences
« Reply #1 on: March 29, 2016, 08:12:12 pm »
 8)

I am surprised that there are issues with OW5!  ;)

Zovacor

  • Full Member
  • ***
  • Posts: 102
    • View Profile
Re: oldWeather teleconferences
« Reply #2 on: March 30, 2016, 04:00:13 am »
Thanks for the updates.

Hanibal94

  • Shipherd
  • Hero Member
  • *****
  • Posts: 4367
  • Better to do it, than live with the fear of it.
    • View Profile
Re: oldWeather teleconferences
« Reply #3 on: March 30, 2016, 07:50:40 am »
Thanks for all the updates!

It's nice to hear that OCR will be used for typed logs in OW6. I was wondering if that was ever gonna happen.

studentforever

  • Editor
  • Hero Member
  • *****
  • Posts: 3268
    • View Profile
Re: oldWeather teleconferences
« Reply #4 on: March 30, 2016, 09:33:40 am »
I fear that humans will still be needed unless OCR has greatly improved for old style typing. I used to volunteer for RNIB and some older books were quicker to type up than scan and put through the OCR program. All, of them, new or old, needed proof reading & correcting before sending out to the customers who would use large print, braille or text-to-speech to access the material.

Caro

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7208
  • Our end is Life. Put out to sea. Louis MacNeice
    • View Profile
Re: oldWeather teleconferences
« Reply #5 on: March 30, 2016, 03:16:48 pm »
Good work Randi.  :)

AvastMH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7157
    • View Profile
Re: oldWeather teleconferences
« Reply #6 on: March 30, 2016, 04:02:39 pm »
Yes -thanks Randi - it is good to know what happens out there in the 'upper reaches'.  :D

Helen J

  • Editor and Shipherd
  • Hero Member
  • *****
  • Posts: 4652
    • View Profile
Re: oldWeather teleconferences
« Reply #7 on: March 30, 2016, 05:08:18 pm »
Thanks for representing us Randi, and for the news.  It's good to have some updates, and to know that our feedback is making a difference.

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 12985
    • View Profile
Re: oldWeather teleconferences
« Reply #8 on: March 30, 2016, 05:12:55 pm »
Thanks, everyone!

Craig

  • Shipherd
  • Hero Member
  • *****
  • Posts: 3360
    • View Profile
Re: oldWeather teleconferences
« Reply #9 on: March 30, 2016, 09:30:08 pm »

I add my thanks too, Randi.

What part of the "system" is being overwhelmed by data in OW4? Surely it can't be storage space. Perhaps there are too many transcribing the same log at the same time?

AvastMH

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7157
    • View Profile
Re: oldWeather teleconferences
« Reply #10 on: March 30, 2016, 10:26:11 pm »
Hi Craig  :)  Bob understands the full depth of it. He discovered that the system prevents more transcriptions once 4,000 entries have been made per log set. I don't know how it got there but it's already holding us up a treat. I was in full flow of dealing with Aaron Dean's handwriting on the John Wells and I've got to wait to get back to it  :'(
I hope our super devs will sort it out soon for us  :D

Bob

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1255
    • View Profile
Re: oldWeather teleconferences
« Reply #11 on: March 30, 2016, 11:54:36 pm »
Hi, Craig -

When a Whaling page is requested for Mark mode, a request goes to the server to fetch the existing annotation data so it can be displayed with the page image. The difficulty is that they have the request size set for a large number of pages rather than just the page in question. Couple that with the increasing size of the requested data set as transcriptions are added (which are also returned with the annotation data), and you get Server Timeout before the data can be assembled and returned. Depending on the average number of marks per page, this will get you to about 90 to 130 pages before the log becomes permanently unavailable for Marking. You can still transcribe, though, since that's apparently managed an item at a time.

- Bob

What part of the "system" is being overwhelmed by data in OW4? Surely it can't be storage space. Perhaps there are too many transcribing the same log at the same time?

Dean

  • Editor
  • Hero Member
  • *****
  • Posts: 2791
    • View Profile
Re: oldWeather teleconferences
« Reply #12 on: March 31, 2016, 12:43:43 am »
Thanks for the update AND for representing us Randi!!!   8)

Sounds like we are becoming (or HAVE become) too famous for words!!   ;)

Great to be part of such a distinguished Group of people! ;)


propriome

  • Shipherd
  • Hero Member
  • *****
  • Posts: 1337
    • View Profile
Re: oldWeather teleconferences
« Reply #13 on: March 31, 2016, 11:03:09 am »
Thank for the updates and for representing us Randi!! :)

mapurves

  • Shipherd
  • Hero Member
  • *****
  • Posts: 1772
    • View Profile
Re: oldWeather teleconferences
« Reply #14 on: March 31, 2016, 04:35:21 pm »
Thank for the updates and for representing us Randi!! :)

Indeed. Your information was worth an upset stomach.