WO361 - Missing Personnel Files - Database Discussion

It's a cunning plan, a plan as cunning as a fox who's just been appointed Professor of Cunning at Oxford University.

Fair enough then Capn' Black Adder, by the way, its rat au van for dinner.

 

http://ww2talk.com/forums/topic/54959-ww2-commonwealth-army-war-diaries/

Further to the posts in the above thread regarding a WO361 database.

I had a good think about a WO361 Database last night while looking at the 4 walls of my digs (not really i was in the pub next door watching the football).

This would a huge undertaking taking years (and then maybe only scratch the surface)

but i came to the view that if your told you need to move a mountain, then there is not much point standing there gawping at it saying it can't be done, you need to start moving the first stones and see where it leads.

Baring a lot of typing (and time), i see it as a win-win situation for myself and others. I get to read through a lot of files and othersget a searchable database which may help them further their research.

Therefore i don't see any reason not to start a database, but would be happier if a few ground rules were sorted and a few concerns addressed before i embark down the road.

Here are a few random thoughts/concerns/preferences of mine regarding a database:-

Legacy & Ownership & Access

I would like any database to be 'freely' available and accessible to all.

Depending on if the database is posted on this site or Lee hosts it on his site, what would happen to it if say this site folded (i know its highly unlikely) or Lee's numbers come up on the Lottery and he retires off into the Sunset? It would be nice to have agreement as to what would happen to the database or who would become its custodian.

How should i treat any files that are passed to me with regard to divulging information within them? i.e. If someone posts a query up on here about a missing soldier do i just say he is mentioned in the file or could i post relevant pages from the files. If i did this then potentially i could be diddling Lee and Andy out of work.
If this database grows, human nature being what it is (this is not directed to anyone on here by the way), someone somewhere will see the potential to earn a quick buck off the back of it. I won't be doing the database with any view for any future financial gain, but IF a potential monetary gain rears its head in the future then i would be happier if this were to happen there was an agreement now that any monies would go to a charity like the RBL.

Format of the database and information covered etc
will only the names of those on the 'missing' rolls be covered? or as Diane refers to them, will it cover the 'incidentals' like names of those that gave testimony or are just referenced within?
Personally i would prefer to include the 'incidentals', but this would be time consuming.
Therefore it may be an idea to just do the rolls first to build up the database quickly to a reasonable size,
then do a second pass to pick up the 'incidentals' at a later date.

Which files to cover/ priority list
Might not be an issue, as it will likely be governed by what files i get sent. Most of the files i've already got are for North Africa & Italy and units specific to my own research.
But its obvious the files likely of most interest will be from the NWE campaign and the more 'fashionable' units.
Personally i don't mind which campaigns or unit files i do, but wouldn't want to concentrate on just one campaign or a narrow scope of units.
My preference would be cover all areas/units proportionally something like as follows:-
For every 3 NWE ones, i do maybe 2 BEF's, and 1 each from remaining campaign areas (Italy, N. Africa, Far East etc) and something similar with regard to which units i.e. for every 4-5 more well known units i do an obscure of lesser known unit.

Just a little food for thought for the mo, but would welcome any further suggestions/ concerns

Will see where the discussion goes and will try to add to it later


Cheers

Scott

 

Hi Scott,

First and most of all, I congratulate you for taking on this project. I have collated two databases over the last seven years and I know just how much work this entails and the stamina required.

Ownership and protection of the database is a difficult one. I agree totally with you that it must be freely available to those who need to access it. I had a decision to make back in 2011 when I was deciding whether to write a book about Chindit 1 or create a website. Obviously I chose option two, with the view that although the information posted on the site is vunerable to 'theft' or misuse by others, there was more scope for the families I wanted to attract to find my work. This decision has proved overwhelmingly to be the correct one for my objectives.

I know that my work has been used 'unoffically' let's say by the medal dealing fraternity. This is not something I am happy with, but comes with the territory of not being able to protect or copyright data on line. It is the price that you have to pay every now and then.

I think you are right to ask the question about posting actual casualty details or pages on the forum. Some families are so delighted to find out information they are not concerned by this, but I have known some to be most put out by this action too. I think you are right to consider our members who perform a copying/research service for others and also that, some people will want to be pointed in the right direction, but would like the thrill of finding and reading the information for themselves.

I think it is as well to add all names invovled in the material, whether they are the casualty or the informant or witness. Countless times in my research and I'm sure in yours, small connections occur from the work of just one witness and a good overall picture can emerge when you cross-reference names, dates and places.

I wouldn't worry about which theatres to concentrate on. As you say this database will build with the input of others and they will send what they have to hand.

I'm also thinking that the WO361 File Reference Excel that Andy posted on to the forum a couple of years back would be a usefull addition at the begining of this thread as a reference tool.

Lastly, I will be glad to help if I can with some WO361 rolls from the Burma theatre. I'm positive others will too, so be prepared!

Best wishes

Steve

 

Thanks Adam & Steve.

With regard to the database i don't think believe the data mining/entry side or proof reading etc would cause any real problem for me.

But the main area of concern would be the initial set up. My excel skills are 'passable', so I would gladly welcome any help available regarding initial formatting and set -up etc.

Once an agreed 'template' was established i can then pootle along to hearts content.

Apologies for only a brief reply but everytime i've go to compile a detailed reply i keep getting 'sidetracked' with work.

Cheers

 

An elaborate index used to be called a calendar at the Essex Record Office. The Imperial War Museum publishes useful summaries of some books and papers it holds: I would call these long lists of subject headings a calendar.

On a computer a search for a name would find the name in a calendar. If I were doing a calendar I would use "Text Edit" because I use a Mac but some might prefer the Microsoft version ( is it Notepad ? ) or Word. I actually prefer Word Perfect for Windows on my 1992 laptop, 96 Kb basic RAM, which still works. Wonderfully clear and fantastic search facility because no box to open. Word Perfect converts to Text Edit with a few extraneous lines.

The term "database" seems to imply a spreadsheet, which probably means a Microsoft program called Excel, or it could be the Mac version called "Numbers", which program seems to involve four times as many bytes as Excel. I use Excel for Mac, but I have two versions, 2007 and 2011. They seem to work together, but for how long ?

I don't like computer spreadsheets but I do use them. However, for a long list of casualties I prefer pieces of A4 paper stuck side by side and I can add dates when I added information much more easily than in a spreadsheet. Note that in an Army service record the date when information was added is extremely useful.

As for The National Archives Copyright Policy, Takedown and Reclosure Policy, Privacy Policy, Record Copying, and User Participation I could make my way through it given several days but the conclusion I would reach is that as long as you say that the information came from Kew and it is Crown Copyright then....( please read the babble produced by Kew, it's all rather circular.)

The best of luck to anybody taking on a new spreadsheet.

That's my five pennies worth.

 

I use Excel 2011 on my Mac too. I find it very easy to use and to sort data quickly and precisely. I'm sure almost all of us have used this system or something similar when decanting paper lists or other data.

Here is one of my smaller examples just to illustrate the point:

 

Thanks, Steve, everybody I know say that spreadsheets and filters are good.

But just think, press F2, type in a few characters, and if it's there the cursor goes straight there. Magic !

John

 

Dear dbf,

I agree with the Oracle, ( oh for the return of Ceefax and Oracle ! ) but the screenshot will please Mrs. papiermache, who will shout QED !

John

 

I've had another think and took on board various suggestions/ideas posted etc and this is my current way of thinking.

I think the 2 main functions of a database should be geared at:-

1. Helping the 'Regimentalist' type researcher compile nominal rolls of battalions etc
2. Provide people researching individual soldiers or relatives with possible further avenues of research

I've also decided that i personally should not divulge any detailed information contained in the files or add any to a database. But, think it maybe
beneficial to give an indication of the type of information contained (i.e. Missing Personnel, provided testimony, listed as potential witness, Incidental reference).
I also think the format, number of fields, information contained should tried to be kept to a minimum

My reasoning behind this is as follows:-
1. Saves time
2. I don't get judged, reprimanded or upset people by divulging what could be sensitive or unpleasant info (The decision to pursue the info will still lie with the individual).
3. I am not potentially diddling people like Andy or Lee out of possible future work research. (and it may even put more work their way).
4. With regard to the 'indication of type of information' it may help people make an informed decision whether to acquire the file or not (either by a trip to Kew or through someones
services)
5. Researchers who already have WO361 files might be happy with names, basic info being added to a DB, but not full details of files that they either spent time or money to acquire. Therefore
they might be more inclined to send me what they have.
6. Still gives the researcher that 'element' of discovery.
7. By keeping required info to a minimum, hopefully more people might feel more inclined to chip in and help along the way.

 

The Actual Database

Main DB pages
Format & information fields

Based on Diane's previous list i see the minimum fields required for the DB as follows:-

1. Surname
2. First Names, Initals etc
3. Army No.
4. Rank
5. Primary Regiment
6. Battalion or Unit
All the above as self explanatory
7. Secondary unit
My reasoning for this is that i've come across a fair amount of correspondance in the files, between various Army departments, trying to trace the whereabouts of men that
have later been transferred to other units. If this info is available it may help in compiling unit rolls etc.
8. Campaign (if required)
9. WO Ref (i.e. WO 361/809 and only this; no page ref's or image no's etc).
In essence, all i think is required is to point a researcher in the right direction, and will save time.
10. Remarks
Generally for highlighting things like inconsistancies (i.e. the same man being referenced with slightly different Army No.s etc)

Front page

Basically a front page listing WO361 files covered by the DB listing following info (which will also help track where i am up to)

1. Wo Ref
2. Regiment(s), units
3. Which battalions/sub-units it covers
4. Extent of Info covered (i.e. All names including incidentals, Just 'missing' roll, file received but not yet transcribed etc)
5. Date added or updated
6. Who or where file originated.
Basically so people can gain accreditation for supplying the file. But, there could be a potential negative side to this, with people constantly getting pm'ed etc for more
information etc. Therefore anyone supplying a file would need to specify if they would require the accreditation or would require their name details witheld.

Producing the DB

Obvious choice is as an excel file xls or similar, mainly because its what i am most comfortable using, and also it can exported/imported/manipulated easily.

Basically, i am hoping to concentrate on the raw data side or things, and leave the technical side of things to the more tech/database savvy people. Therefore if anyone wants
to look into things like search forms, data manipulation etc then be my guest.


So.... for the moment thats my twopenneth worth.

If anyone has further suggestions, ideas or areas of concern then let me know.

Also, i wouldn't mind a bit more help, ideas and guidance to the 'Ownership & Protection' side of things as Steve previously mentioned.

And finally if anyone fancies knocking up a basic excel template based on the above then then feel free

Cheers

 

Hi everyone, my first questions would be from a long term and volume perspective. Excel spreadsheets can rapidly become bogged down during searches when their data limits are pushed. If it is just a flat spreadsheet - no references, links to other sheets with more detail etc it may support a great deal of data, but it also may not be the best solution for long term use, growth, and sharing.

One should seriously look at developing a true database backend, with a web based front end. This would allow significant growth, ability to search effectively, capability to add different levels of access (basic info for some, more detailed for others, full details for maintainers etc) using security models, as well as a standardized interface.
This would also take the load off the local machine, where search time and capability is restricted by local resources and place it on a server where more resources are available. It would also allow for better protection of the data through redundancy and backups provided at the server level. Access to multiple users, admins, and maintainers could also be provided simply. Images could be locally stored and linked from the individual records.

With a web based front end a publicly available search function could also be provided, themed to WW2Talk, and linked from the forum.

I would take a bit more to design and setup - but well worth it in the long run IMHO.

As I've previously said, I can donate a domain name and the necessary hosting for the site as well as daily backup. I would also be willing to be involved in the design of the back and front end.

Wayne

 

Diane

Thanks for the xls

Wayne

The one glaring weakness for the producing a database would be my lack of knowledge of the technical side regarding databases, interfaces, hosting, search facilities etc. Therefore i will take any help i can get.

If you are prepared to be involved with the front & back end design, hosting etc then it would be greatly appreciated.

Cheers

 

Scott,

Been looking through my files to see what I have. More than I first thought really. 20 sets of MIA Rolls from various WO361 files, mostly Burma. Some typed, some in good readable order and some handwritten and not so easily read. I'm sure you have seen a fair amount of examples yourself.

A question for you. Would you be interested in men who after being classified as MIA, were later to be found as POW's. I ask this because of my work on the prisoners of Rangoon Jail during WW2. Here I can decant the info from my own Excel and therefore data may be able to be copied in very straightforwardly.

Steve

 

Steve

I have been looking through my own WO 361 files and there are loads of men posted MIA that turn up as POW's, along with correspondance from various Army departments, Red Cross etc giving camp names or details.

My view is if a researcher gets pointed in the right direction and finds these additional details it will help fill in the gaps and give additional avenues of research. Also the more names that get added, then the more likely the DB becomes the first point of reference for someone compiling a 'nominal' list of a unit.

Therefore, i would definately be interested in your Rangoon prisoner lists. Only issue i see is backtracking the men listed to the appropriate WO 361 file reference (unless you've already done it elsewhere)

Cheers