Author Topic: Ice Hunting Guide - Appendices  (Read 106 times)

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 11770
    • View Profile
« Last Edit: January 13, 2018, 03:57:37 pm by Randi »

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 11770
    • View Profile
Re: Ice Hunting Guide - Appendices
« Reply #1 on: July 17, 2016, 10:25:12 pm »
Automated Download

Courtesy of Michael (mapurves) who explained it so clearly that even I managed to do it ;)

There are three elements:
  • Wget.ahk
    Copy the Wget.ahk text below into Notepad (or something similar) and then save it on your hard disk as Wget.ahk.
    For the example below, you would save it in C:\Users\rcl\Desktop\OW Data\Hot Keys.
  • Wget_Config.txt
    Copy the Wget_Config.txt text below into Notepad (or something similar), edit it as necessary, and then save it on your hard disk as Wget_Config.txt.
    This should be stored in a subdirectory, named Config, of the directory that contains Wget.ahk.
    For the example below, you would create it in C:\Users\rcl\Desktop\OW Data\Hot Keys\Config.
  • wget.exe
    Download wget.exe from https://eternallybored.org/misc/wget/current/wget.exe
    Michael used the 32-bit binary version.
    This should be moved to the directory that contains Wget.ahk.
    For the example below, you would move it to C:\Users\rcl\Desktop\OW Data\Hot Keys.



Wget.ahk

This should not need editing unless you want to customize it.

Quote
#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
; SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.

;  This script will use Wget to download the images for transcribing. It reads a
;  config file for the necessary parameters. It prompts the user for the first
;  image to download, the number of images to download and whether or not to
;  enhance the images to improve the readability. It also prompts the user to
;  delete the previous images. If images are enhanced, the enhanced image will have
;  the letter a appended to the name: i.e. Image_0015.JPG and Image_0015a.jpg.

;  Author: M. A. Purves    11 Jan 2017
;  Updates:
;          M. A. Purves    27 Feb 2017. Search through the browser tabs to find
;                          the tracking page, if it is there.
;          M. A. Purves    04 Jan 2018. Split the delete images and xml files into
;                          two separate questions. If there is no XML_Dir in the
;                          config file, the question about deleting XML will not
;                          be asked.
;          M. A. Purves    05 Jan 2018. Use StringGetPos to get the image name.
;                          I clear the clipboard at the start and the program
;                          exits if nothing is copied or if the copied items
;                          are not valid image URLs.
;                          I used Randi's copying instructions and added her
;                          her exitapp key.

SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.
SetTitleMatchMode, 2

Batch_File  = %A_ScriptDir%\Wget.bat
Config_File = %A_ScriptDir%\Config\Wget_Config.txt

Debug          :=   0     ; If we want to run the batch file manually after an edit.
Chime           = *16     ; Warning sound for warning messages.
Delete_Images  :=   0     ; Initialize just in case
Delete_XML     :=   0     ; Initialize just in case
Enhance_Images :=   0     ; Initialize just in case
Split_Images   :=   0     ; Initialize just in case
Clipboard       =         ; Clear the clipboard
CRLF           = `r`n     ; DOS needs a carriage return and a line feed.
Bright         =       ; Turn on bright to enhance an echo command
Normal         =       ; Turn off bright

#w::ExitApp  ; Assign a hotkey to terminate this script.
             ; # is the 'Windows' key.

!6::                      ; We run the script  (6 is a temporary key!)
   GoSub Read_Config      ; We read the config file
   if (! OK)
     Return

   WinActivate %BrowserName%
   Sleep 250

   GoSub Get_Tracker

   MsgBox, 4096, Get Images,
   (
Copy the URLs of the images you wish to download.
Then click OK.
   )
   if (! Clipboard)
      {
      MsgBox No image URLS were copied. Exiting program.
      exit
   }
   No_files := 0                    ; We get the number of files.
   Loop, parse, clipboard, `n,`r
      {
      No_files += 1
   }

   MsgBox, 4, Delete?, Do you want to delete the old images?
   IfMsgBox Yes
      Delete_Images := 1
   Else
      Delete_Images := 0

 
   if (XML_Dir)
   {
      MsgBox, 4, Delete?, Do you want to delete the old XML files?
      IfMsgBox Yes
         Delete_XML := 1
      Else
         Delete_XML := 0
   }

  GoSub Write_Batch

   if (! OK)
      Return

   if (! Debug)
      Run, %Batch_File%
Return

; =========================== Subroutine Read_Config  =============================
;    We read the configuration data from the config file.
; =================================================================================

Read_Config:
   OK := 1
   IfNotExist, %Config_File%
   {  SoundPlay %Chime%
      MsgBox,4096,No File, Sorry, there is no file %Config_File%! This script will exit.
      OK := 0
      Return
   }
   Loop, Read, %Config_File%
   {
      StringMid, Char, A_LoopReadLine, 1, 1
      if (Char != ";") {                            ; We skip comment lines.
         StringGetPos, pos, A_LoopReadLine, =
         StringLen, length, A_LoopReadLine
         StringLeft,  VRBL, A_LoopReadLine, pos
         StringRight, VAL,  A_LoopReadLine, length-pos-1
         %VRBL% := VAL
      }
   }   ; End Loop

   Iview_Opts := ""                                 ; We start with nothing.
   if (Gray)
      Iview_Opts = %Iview_Opts% /gray
   if (Contrast)
      Iview_Opts = %Iview_Opts% /contrast=%Contrast%
   if (Gamma)
      Iview_Opts = %Iview_Opts% /gamma=%Gamma%
   if (Enhance_Image or Split_Image)
      {
      Original_Dir = %Image_Dir%\Originals
      Event_Dir    = %Image_Dir%\Events
   } else {
      Original_Dir = %Image_Dir%
   }
Return

; =========================== Subroutine Write_Batch  =============================
;    We write the batch file.
; =================================================================================

Write_Batch:
   file := FileOpen(Batch_File, "w")
   if !IsObject(file)
   {
      SoundPlay, %Chime%
      MsgBox Can't open "%Batch_File%" for writing.
      OK := 0
      Return
   }
   CMD = ECHO ON
   GoSub Write_CMD
   if (Debug) {                ; This prevents an accidental running.
      GoSub Run_Safely         ; The user checks the file, removes the
   }                           ; EXIT at the start and runs it manually.

   if (Delete_Images or Delete_XML)
      {
      GoSub Delete_Files
   } else {
      MSG = Keeping old image and XML files.
      GoSub Write_Bright
   }

   CMD   = CHDIR %Original_Dir%
   GoSub Write_CMD

   CMD  = DEL %LOG%             ; We delete the previous log file.
   GoSub Write_CMD

   Numb := 1                    ; We process the images in the clipboard.
   Loop, parse, clipboard, `n,`r
      {
      GoSub Parse_Clipboard
 
      if (! Image)              ; The clipboard had no JPGs.
         Return

      MSG = Processing file %Numb% of %No_Files%
      GoSub Write_Bright
      CMD = %WGET% %Wget_OPTS% -O%Original% -a%LOG% %URL%   ; Get the image
      GoSub Write_CMD

      if (Enhance_Image and Iview_Opts) { ; We enhance the image.
         GoSub Enhance_Image
      }
      if (Split_Image) { ; We split the Event Page from the image.
         GoSub Split_Image
      }
      Numb += 1
   }   ; End Loop

   CMD = EXIT
   GoSub Write_CMD
   file.Close()
Return

; =========================== Subroutine Write_CMD  ===============================
;    We write the batch file command. We sometimes need a wee sleep.
; =================================================================================

Write_CMD:
   Line = %CMD% %CRLF%
   File.Write(Line)
   Sleep 50
Return

; =========================== Subroutine Write_Bright =============================
;    We write a bright command, turning ECHO OFF and ON around the command.
; =================================================================================

Write_Bright:
   CMD  = ECHO OFF
   GoSub Write_CMD
   CMD  = ECHO %Bright% %MSG% %Normal%
   GoSub Write_CMD
   CMD  = ECHO ON
   GoSub Write_CMD
Return

; =========================== Subroutine Get_Tracker  =============================
;    We look for a browser tab open on the Tracker site. If there, we
;    activate it. If not, we open the Tracker site on a new tab.
; =================================================================================

Get_Tracker:
   Found_tracker := 0
   Quit          := 0
   Count         := 1

   while (! Found_tracker && ! Quit)
      {
      ifWinExist, Tracker
         {
         WinActivate, Tracker          ; Found it.
         Sleep 100
         Found_Tracker := 1
         Quit          := 1
      } else {
         WinGetTitle, Title, A          ; Title of open browser tab.
         Sleep 100
         if (Title == First_Title)      ; Checked them all. No luck.
            {
            Quit := 1
         } else {
             if (Count == 1)
               First_Title = %Title%
            Count += 1                   ; Go to the next tab.
            Send ^{Tab}
            Sleep 200
         }
      }
   }

   If (! Found_Tracker)
         Run, %BROWSER% %TRACKING%
   Sleep 1000
Return

; =========================== Subroutine Enhance_Image  ===========================
;    We enhance the log page image and create an Event page image.
; =================================================================================

Enhance_Image:
   ; We enhance the original log page image.
   CMD = %IVIEW% %Original% %Iview_OPTS% /convert=%Image%
   GoSub Write_CMD
Return


; =========================== Subroutine Split_Image  ===========================
;    We create an Event page image.
; =================================================================================

Split_Image:
   ; Go to the image directory and copy the event section out of
   ; of the image and save it in the Events directory.

   CMD = CD %Image_Dir%
   GoSub Write_CMD
   SetFormat, Float, 4.0
   Image     = %Base%.jpg
   StartX   := ImageW/2          ; We take the right half of the image.
   EndX     := ImageW/2

   CMD = %IVIEW% %Image% /crop=(%StartX%, 0, %EndX%, %ImageH%, 0) /convert=%Event%
   GoSub Write_CMD

   CMD = CD %Original_Dir%        ; Go back to the original dir for the next one.
   GoSub Write_CMD
Return

; =========================== Subroutine Delete_Files  ============================
;    We delete the old files.
; =================================================================================

Delete_Files:
   if (Delete_Images)
      {
      CMD = DEL %Image_Dir%\*.jpg
      GoSub Write_CMD
      MSG = Deleting old image files.
   }

   if (XML_Dir and Delete_XML)     ; If in the config file, we delete them.
      {
      CMD = DEL %XML_Dir%\*.xml
      GoSub Write_CMD
      MSG = Deleting old xml files.
   }
   GoSub Write_Bright

   if (Iview_Opts and Delete_Images)  ; Delete these ones too.
      {
      CMD = DEL %Original_Dir%\*.jpg
      GoSub Write_CMD
      CMD = DEL %Event_Dir%\*.jpg
      GoSub Write_CMD
   }
Return

; =========================== Subroutine Parse_Clipboard ==========================
;    We parse a line in the clipboard to get the image name.
; =================================================================================

Parse_Clipboard:
   Found := 0
   URL   := A_LoopField
   StringGetPos, ImagePos, URL, / , R
   Image := SubStr(URL, ImagePos+2)
   StringGetPos, pos, Image, .
   StringLeft,  Base, Image, pos
   if (Enhance_Image or Split_Image)
      {
      Image     = %Image_Dir%\%Base%.jpg
      Event     = Events\%Base%e.jpg
      Original  = %Original_Dir%\%Base%o.jpg
   } else {    ; No enhancing. The original is the image.
      Original  = %Image_Dir%\%Base%.jpg
   }
   if (! Image)
      {
      MsgBox There is no image in the clipboard
      file.Write("EXIT`r`n")
      file.Close()
      exit
   }
Return

; =========================== Subroutine Run_Safely ===============================
;    We put an exit at the start. The user edits the file, removes the exit
;    command and runs it manually.
; =================================================================================

Run_Safely:
      MSG = Debug Mode. Edit this batch file then run it.
      GoSub Write_Bright
      CMD = EXIT
      GoSub Write_CMD



Wget_Config.txt

The parts you need, or may need, to edit are red.
I chose to comment out the XML_Dir line because I am afraid of deleting the files by accident.

Quote
;
;  This file contains the variables needed do directly download
;  ship images using WGET, and it contains the variables needed
;  to enhance the images for better readability.
;
;  The image directory is the same as the Local Image Dir: in
;  the Calc Spreadsheet setup page.
;
Image_Dir="C:\Users\rcl\Downloads"
;
;  The XML directory is the same as the User Dir: in
;  the Calc Spreadsheet setup page. If you wish to have
;  old XML files deleted, leave this line in, otherwise
;  comment it out, or delete it.
;
;XML_Dir="C:\Users\rcl\Desktop\OW Data"
;
; Put the path to your browser here.
;
BROWSER="C:\Program Files (x86)\Mozilla Firefox\firefox.exe"
BrowserName=Firefox
;
; This example uses the Burton Island 1955 tracker
TRACKING=https://docs.google.com/spreadsheets/d/1dJcQN3FfqIJSiTvrniw_8snQjX8Qx36KeUMYse2Q3fA/edit#gid=2056216563
;
; Get wget.exe from https://eternallybored.org/misc/wget/current/wget.exe
; I used the 32-bit binary version.
;
WGET="C:\Users\rcl\Desktop\OW Data\Hot Keys\wget.exe"
;  --secure-protocol=TLSv1
;   --no-check-certificate
Wget_OPTS=-nc -nv
LOG=Wget.log
;---------------------------------------------------------------
; If the WR and Event pages are contained in one image and
; if you wish them to be cut so the WR and Event pages are
; in separate images, set Split_Image to 1.
;---------------------------------------------------------------
Split_Image=0
;---------------------------------------------------------------
; If you want the images to be enhanced for better readability
; set Enhance_Image to 1.
;---------------------------------------------------------------
Enhance_Image=0
;---------------------------------------------------------------
; If you wish images to be enhanced or split, you need IVIEW.
;---------------------------------------------------------------
IVIEW="C:\Program Files (x86)\IrfanView\i_view32.exe"
;
; The enhancement options. Set to 0 if some are not wanted.
;
Gray=1
Contrast=33
Gamma=0.35
;
; For splitting images so the WR and Events are on separate images.
; Enter the image size of an original image in pixels.
; Get the image size from Irfanview Image/Information.
;
ImageW=5616
ImageH=3744





Once you have these three files...

  • Ensure your browser is open, start Wget.ahk and enter Alt-6.
     
  • The tracker you selected will be opened in the browser you selected and you should see something similar to this:

     
  • Go to the tab for the log pages you are downloading, select the links for the pages want to download and click Ctrl+C to copy the links.

     
  • Click OK.



You may find that you need to do the following to make the file executable:
  • Right-click on Wget.ahk in the file manager
  • Select Open with from the menu that pops up.
  • Select Choose another app from the next menu that pops up.
  • Select AutoHotkey Unicode 64-bit and Always use this app to open .ahk files.





If you have questions or problems, please send me a PM,
Randi
« Last Edit: January 07, 2018, 02:29:20 am by Randi »

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 11770
    • View Profile
Re: Ice Hunting Guide - Appendices
« Reply #2 on: July 20, 2016, 10:45:17 pm »
Adding Times to the Event Input Wizard

If your log book does not use our default times or time ranges, you can add to the default list.

Make sure you have a backup!
I have had LibreOffice crash several times when I was tying to do editing.
However, that may have been because I tried to look at a log page while editing.



Select: Tools -> Macros -> Edit Macros

In the new window, select your current spreadsheet name and then select Standard.
Burton_USN_1955_4 - Working.ods is shown here. Your file name will differ.

Double click EventInput or select the EventInput tab in the bottom row of the window.

Click in the Time box.



Click on the down arrow (circled) to open the drop-down list for the "List entries" field.
Then use the keyboard arrows to position the cursor where you want to add the new times.
(Clicking in the list and using the scroll-bar don't work for me. This appears to be a LibreOffice bug/feature.)

To add at the end of the list or within the list:
Position the cursor at the end of a line, press Shift+Enter, and then type a time/time-range (e.g., 00 to 04).
To add at the beginning of the list:
Position the cursor at the beginning of the first line, type the time/time-range, and then finish with Shift+Enter.



When you have finished adding times, hit Enter and then close the window.



You can edit the times for the Location Wizard in a similar manner.

Save your spreadsheet after making the changes!
Also, you'll need to repeat your changes for any new versions of the spreadsheet that you download.



Caution!
Some changes can have unintended side-effects.
Before making any other changes, please ask in the forum or PM a moderator.
« Last Edit: January 13, 2018, 03:56:25 pm by Randi »

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 11770
    • View Profile
Re: Ice Hunting Guide - Appendices
« Reply #3 on: July 21, 2016, 10:30:12 pm »
Reserved
« Last Edit: January 05, 2018, 11:39:37 pm by Randi »

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 11770
    • View Profile
Re: Ice Hunting Guide - Appendices
« Reply #4 on: July 22, 2016, 11:12:49 pm »
Reserved
« Last Edit: January 05, 2018, 11:39:47 pm by Randi »

Randi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 11770
    • View Profile
Re: Ice Hunting Guide - Appendices
« Reply #5 on: July 23, 2016, 10:23:41 pm »
Reserved
« Last Edit: January 05, 2018, 11:39:57 pm by Randi »