Battling with Picker Automation
The UIPickerView is the iOS equivalent of a dropdown list.
Appium and its Python WebDriver bindings have failed to support picker interactions on iOS.
This is my solution to automating UIAPicker and UIAPickerWheel interactions. Here, I find a picker on the screen and select a value from it using the select_animal(self, index) method.
class PickerPage: def __init__(self, driver): self.driver = driver self.picker_values = {} # Actions def select_animal(self, index): # Get the animal picker on screen if not self.is_animal_selection_on_screen() and not self.is_animal_picker_active(): # The animal button brings up the animal picker self.animal_button().click() picker = self.animal_picker() self.move_to_picker_value(picker, index) self.picker_done_button().click() def move_to_picker_value(self, picker, index): self.return_to_first_picker_value(picker) wheel = self.get_picker_wheel(picker) while index >= self.get_current_picker_position(wheel): self.tap_next_value(picker) def return_to_first_picker_value(self, picker): wheel = self.get_picker_wheel(picker) current_position = self.get_current_picker_position(wheel) # Return to position 1 while current_position > 1: self.tap_previous_value(picker) current_position = self.get_current_picker_position(wheel) # Booleans def is_animal_selection_on_screen(self): return self.animal_button().is_displayed() def is_animal_picker_active(self): try: self.animal_picker() return True except NoSuchElementException: return False # Object Retrieval def animal_button(self): return self.driver.find_element_by_name("animalButton") def animal_picker(self): return self.driver.find_element_by_name('animalPicker') def picker_done_button(self): return self.driver.find_element_by_name('doneButton') def get_picker_wheel(self, picker_view): return picker_view.find_element_by_class_name('UIAPickerWheel') # Picker Position def get_current_picker_position(self, wheel): return self.get_picker_position(wheel)[0] def get_max_picker_position(self, wheel): return self.get_picker_position(wheel)[1] def get_picker_position(self, wheel): pattern = r".+, (\d) of (\d)" string = wheel.get_attribute('value') match = re.match(pattern, string) current_position = int(match.group(1)) max_position = int(match.group(2)) return current_position, max_position # Picker Interactions def tap_next_value(self, picker): def tap_picker(self, picker, 44) def tap_previous_value(self, picker): def tap_picker(self, picker, -44) def tap_picker(self, picker, y_offset_from_picker_centre): height = picker.size['height'] width = picker.size['width'] x_offset = width / 2 y_offset = ((height / 2) + y_offset_from_picker_centre) move_action = ActionChains(self.driver).move_to_element_with_offset( picker, x_offset, y_offset ).click() move_action.perform()
As you can see, there's a lot behind the curtain of that simple-sounding select_animal(self, index) method. Pretty horrendous, huh?
Rather than walk through the code above step by step, I'll focus on the core parts of my interaction with the picker elements.
Activating the picker is a straight-forward click() on a UIButton element. If the button isn't on-screen at the time, it will be scrolled into view before the click is executed*. Nothing particularly taxing.
To Appium, a picker consists of more than one element.
All pickers have a:
UIAPicker - the view which defines the touchable area of the picker. Parent of UIAPickerWheel.
UIAPickerWheel - the view which shows the currently selected value, and contains information about the position of the picker.
Some pickers also have:
A 'Done' button of some variety, to dismiss the picker.
Auxiliary buttons to help you with your selection, or to assist you in navigating around the form.
Understanding the Picker's State
Since there's no accessible 'position' property on picker elements, the process for gaining an understanding of the state of the picker is rather convoluted.
UIAPicker only ever has one child UIAPickerWheel, but the attribute values associated with the UIAPickerWheel element change as you scroll through the picker. Therefore, UIAPickerWheel is not a stable element, but UIAPicker is.
In order to gain access to the state information held by the UIAPickerWheel element, find the parent UIAPicker using its name attribute, then find a child UIAPickerWheel element by searching by class name:
wheel = self.driver.find_element_by_name( 'animalPicker' ).find_element_by_class_name( 'UIAPickerWheel' )
As seen in get_picker_position(self, wheel) above, you can then access the value of the wheel element, which will get you a string containing the text of the currently selected value on the picker, and the position of that value in the picker. If the second of five elements, 'Cat', was selected on the picker, the value of the wheel would be "Cat, 2 of 5". By capturing** the two numbers in the string and converting them to integers, you can gain an understanding of the current position of the picker.
Changing the Picker's Selected Value
Now that you understand where you are in the picker, you might want to change the selection.
Unfortunately, while you as a user may be able to see the surrounding picker values on the screen, Appium is blind to every option except the currently selected one, so you're going to have to tell Appium where to touch the screen, relative to the currently selected value, as seen in tap_picker(self, picker, y_offset_from_picker_centre) above.
For reference, the (x, y) position (0, 0) is at the top left corner of the screen; increasing x will move you to the right across the screen and increasing y will move you down the screen (somewhat counter-intuitive).
Accessing the size property of the picker yields information about the size of the view. The currently selected value will be vertically centred in the view. As a good starting point, find the centre of the picker by halving the height and width of the view.
In their iOS Human Interface Guidelines, Apple recommend that tap targets have a minimum height and width of 44 points.
Make it easy for people to interact with content and controls by giving each interactive element ample spacing. Give tappable controls a hit target of about 44 x 44 points.
Unless your app is modifying the height of each value in the picker, you can bet*** on tapping 44 points above/below the centre of the selected value and hitting the next one in the list. Use an ActionChains object to move 'the cursor' to where you want to tap, then execute a click at the cursor's current position and perform the action chain.
ActionChains(self.driver).move_to_element_with_offset(picker, x_offset, y_offset).click().perform()
In the line of code above, move_to_element_with_offset(to_element, xoffset, yoffset) can be explained in three stages:
Move the cursor to the top left corner of to_element (the first argument, a WebElement).
Move the cursor from its current position to the right by a magnitude of xoffset (int) points.
Move the cursor from its current position downwards by a magnitude of yoffset (int) points.
Note: To have the cursor move left or upwards from its current position, pass in a negative value to xoffset or yoffset.
After performing the chain of actions, the picker selection will (hopefully) change and if needed, you can begin the cycle again to assert state or change your selection.
They never looked like they'd be this much trouble.
As part of the click() method, a scrolling touch event will be executed to bring the element into view if it is not already on-screen. The exception to this is if the touch event is intercepted or picked up by another view. This might be the case if you have a UIScrollView within a UIScrollView, which reacts to the touch event instead of the parent UIScrollView containing your element to be clicked. In this situation, you can scroll the element into view using the drag_and_drop(source, target) API before clicking on the element.
** I wrote a post last week on working out how to use regular expression matching in Python. That was how I figured out that the regex I needed to extract the numbers from the value of the picker wheel was r".+, (\d) of (\d)".
*** Profit not guaranteed. All bets are subject to the possibility of failure.
