Import Equipment Structure

[caseywatts]

If we have an excel spreadsheet with a bunch of information about Equipment Items (already organized into Models and Categories), we should be able to import that information into Reservations.

Kelly Barrick requests that we help her import from another database. She'll be able to coerce her data into a csv file if we tell her what it should look like - ideally the app would be able to take it from there.

We have code in import_users_controller which should be quite similar!

[orenyk]

I've started building the framework for an import_equipment_models_controller but just realized after re-reading the issue description that it sounds like you want to import EquipmentObjects, not just EquipmentModels. This is totally do-able, just not what I've been working on :-P. Let's discuss on Wednesday.

TO DO:

• finish writing import_equipment function in lib/csv_import.rb
• figure out if we need to move this into separate files? (getting messy)
• we need to take category names and convert them to IDs before creating / updating the equipment...
• also write up code to create a new category if none exists
• ensure that import page for equipment clarifies that the description cannot have any commas in it
• check route naming and possibly refactor since the user import functionality was not named with other import functionality in mind
• probably much much more :-/

[orenyk]

This appears to be somewhat trickier than importing users since we have a series of related models whose associations have to be set up correctly. Luckily, since both EquipmentModels and Categories enforce uniqueness on their names, it will be easy to look up their IDs based on a given name. That said, I'm not sure what to do if a user gives us an EquipmentObject with an EquipmentModel (or, less problematically, a Category) that doesn't exist. Categories are easy, they only need a name for creation, although a user might want to input the relevant preferences during the import phase if not specified in a CSV file.

At this point, I feel like we should ask users to give us 3 separate CSV files, one for each model (perhaps with the EquipmentModel and Category files being optional). That way we can run the imports in sequence - first Categories, then EquipmentModels, and finally EquipmentObjects, ensuring that all of the ActiveRecord associations are set up correctly. I'm going to start fleshing out this functionality and we'll see how it goes.

EDIT: I just realized that we could just separate this into three separate actions and make all of them optional; basically, you give us files for whatever new data you want to import, and we'll run the actions for which we have CSV files. I hope that makes sense, but I think this will come together nicely. I'm going to also move the model-specific import functions out the lib/csv_import.rb file since it was getting super cluttered.

[orenyk]

Ok, so I've started implementing the aforementioned strategy. I've set up a new controller and put together the /import_equipment/import view with example CSV files for clarification. Here's how I anticipate the controller functioning:

1. Check whether or not a CSV file was uploaded for a specific import type
2. If it was, process the CSV file and validate the output
3. Assuming valid output, attempt to import the processed data (taking in an overwrite option for existing data)
   • this is where we take Category/EquipmentModel names and convert them to IDs
4. Return the import results and move on to the next import type (Categories -> EquipmentModels -> EquipmentObjects)
5. After finishing with the EquipmentObjects, report the results of each requested import type

I think this is all doable, and will get on this next time (moving on to #484).

[orenyk]

Talk to Austin about where to put all this code - see comments in import_equipment_controller.rb for now. Long story short - this all has to be moved to libraries; one for general CSV stuff, one for user stuff, and one for equipment stuff. Also, write tests for all libraries!

[orenyk]

All of the equipment importing methods are now contained in lib/import_equipment.rb and everything seems to be working. I still have to create the import status view and then write tests for EVERYTHING. I probably should have written the tests first, but oh well...

[orenyk]

Added import status view and associated partials, as well as links to the import page on all equipment index views. We need to test everything, including the library, and also consider adding the sort_order as an import-able parameter of categories.

[orenyk]

I've added sort_order as an importable parameter for categories and updated the views and validation functions accordingly. I also created the files for testing but will work on tests next time.

[orenyk]

Finally got some basic test files set up and configured RSpec to work with them. I wrote tests to check for the various methods of the EquipmentImport module and pending tests for the module's functionality. I've also created a file with pending tests for the CsvImport module, although I'll probably only write tests for the basic CSV functionality (not the user import stuff). Up next is writing tests, including dummy data.

[orenyk]

I started working through the category import functionality and writing tests. There was some nifty Ruby code to replace a valid header with an invalid header, but otherwise I'm having major issues. The validation functions are writing to the flash which is not accessible from the tests... I'm not sure if I'm just writing the wrong tests or if the code is bad. Will review with @austinczarnecki or @caseywatts.