The Battle for Wesnoth  1.13.10+dev
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Modules Pages
Classes | Typedefs | Functions
utf8 Namespace Reference

Functions for converting Unicode wide-char strings to UTF-8 encoded strings, back and forth. More...

Classes

class  invalid_utf8_exception
 Thrown by operations encountering invalid UTF-8 data. More...
 

Typedefs

typedef ucs4::iterator_base
< std::string,
ucs4_convert_impl::convert_impl
< char_t >::type > 
iterator
 
typedef char char_t
 
typedef std::string string
 

Functions

static int byte_size_from_utf8_first (const unsigned char ch)
 
utf8::string lowercase (const utf8::string &s)
 Returns a lowercased version of the string. More...
 
size_t index (const utf8::string &str, const size_t index)
 Codepoint index corresponding to the nth character in a UTF-8 string. More...
 
size_t size (const utf8::string &str)
 Length in characters of a UTF-8 string. More...
 
utf8::stringinsert (utf8::string &str, const size_t pos, const utf8::string &insert)
 Insert a UTF-8 string at the specified position. More...
 
utf8::stringerase (utf8::string &str, const size_t start, const size_t len=std::string::npos)
 Erases a portion of a UTF-8 string. More...
 
utf8::stringtruncate (utf8::string &str, const size_t size)
 Truncates a UTF-8 string to the specified number of characters. More...
 
void truncate_as_ucs4 (utf8::string &str, const size_t size)
 Truncates a UTF-8 string to the specified number of characters. More...
 

Detailed Description

Functions for converting Unicode wide-char strings to UTF-8 encoded strings, back and forth.

Typedef Documentation

typedef char utf8::char_t

Definition at line 28 of file unicode_types.hpp.

Definition at line 41 of file unicode.hpp.

typedef std::string utf8::string

Definition at line 29 of file unicode_types.hpp.

Function Documentation

static int utf8::byte_size_from_utf8_first ( const unsigned char  ch)
static

Definition at line 36 of file unicode.cpp.

References preferences::editor::TransitionUpdateMode::count, and count_leading_ones().

Referenced by index(), and size().

utf8::string & utf8::erase ( utf8::string str,
const size_t  start,
const size_t  len = std::string::npos 
)

Erases a portion of a UTF-8 string.

Parameters
strUTF-8 encoded string.
startStart position.
lenNumber of characters to erase.
Note
This implementation does not check for valid UTF-8. Don't use it for user input.

Definition at line 104 of file unicode.cpp.

References index(), and size().

Referenced by gui2::password_box::delete_selection(), gui2::text_box::delete_selection(), and truncate().

size_t utf8::index ( const utf8::string str,
const size_t  index 
)

Codepoint index corresponding to the nth character in a UTF-8 string.

Returns
str.length() if there are less than index characters.

Definition at line 71 of file unicode.cpp.

References byte_size_from_utf8_first(), ERR_GENERAL, and i.

Referenced by pathfind::a_star_search(), unit::ability_active(), display::action_pressed(), actions::actor_sighted(), gui2::tree_view_node::add_child(), simple_wml::node::add_child_at(), config::add_child_at(), simple_wml::node::apply_diff(), config::apply_diff(), BOOST_AUTO_TEST_CASE(), team::calculate_enemies(), team::calculate_is_enemy(), editor::editor_controller::can_execute_command(), play_controller::hotkey_handler::can_execute_command(), ai::cfun_ai_get_suitable_keep(), wesnothd::server::cleanup_game(), config::clear_diff_track(), persist_file_context::clear_var(), gui2::chatbox::close_window_button_callback(), ng::connect_engine::connect_engine(), gui2::text_box_base::copy_selection(), create_cursor(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::create_item(), ng::create_engine::current_extra(), gui2::dialogs::unit_attack::damage_calc_callback(), gui2::dialogs::game_load::delete_button_callback(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::delete_item(), gui2::dialogs::unit_recall::dismiss_unit(), display::enable_menu(), erase(), gui::menu::erase_item(), editor::editor_controller::execute_command(), play_controller::hotkey_handler::execute_command(), gui2::dialogs::mp_join_game::fetch_game_config(), pathfind::find_routes(), ng::flg_manager::find_suitable_faction(), font::pango_text::format_link_tokens(), mp::game_info::game_info(), play_controller::hotkey_handler::get_action_image(), editor::editor_controller::get_action_state(), tod_manager::get_area_by_index(), gui2::styled_widget::get_canvas(), gui2::tree_view_node::get_child_at(), font::pango_text::get_column_line(), tod_manager::get_current_area_time(), image::cache_type< T >::get_element(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::get_item_shown(), wb::side_actions::get_numbers(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::get_ordered_index(), gui2::addon_list::get_selected_addon(), team::get_side_color_index(), team::get_side_color_range(), font::pango_text::get_token(), gui2::dialogs::addon_manager::get_type_filter_visibility(), campaignd::server::handle_delete(), gui2::dialogs::preferences_dialog::hotkey_type_filter_callback(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::impl_create_items(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::impl_draw_children(), insert(), replay_recorder_base::insert_command(), spirit_po::catalog< hashmap_type, pf_compiler >::insert_message(), font::pango_text::insert_text(), game_lua_kernel::intf_get_sides(), intf_music_add(), team::is_enemy(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::item(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::item_ordered(), team::knows_about_team(), wesnothd::make_add_diff(), wesnothd::make_change_diff(), wesnothd::make_delete_diff(), ng::depcheck::MAKE_ENUM(), terrain_filter::match_internal(), display::menu_pressed(), variable_info_implementation::replace_range_h::operator()(), variable_info_implementation::merge_range_h::operator()(), recall_list_manager::operator[](), game_events::event_handlers::operator[](), cave_map_generator::cave_map_generator_job::place_chamber(), gui2::pane::place_children(), gui2::pane::place_or_set_origin_children(), ai::readonly_context_impl::power_projection(), gui2::dialogs::mp_create_game::pre_show(), turn_info::process_network_data(), wesnothd::game::process_turn(), ai::push_move_map(), metrics::record_sample(), simple_wml::node::remove_child(), config::remove_child(), config::remove_children(), replay_recorder_base::remove_command(), ai::ai_default_rca::candidate_action_evaluation_loop::remove_completed_cas(), wesnothd::server::remove_player(), hotkey::remove_wml_hotkey(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::select_item(), editor::map_context::set_active_area(), ng::create_engine::set_current_era_index(), ng::flg_manager::set_current_faction(), ng::flg_manager::set_current_gender(), ng::flg_manager::set_current_leader(), ng::create_engine::set_current_mod_index(), gui2::dialogs::editor_resize_map::set_direction_icon(), editor::editor_palette< Item >::set_group(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::set_item_selected(), gui2::generator< minimum_selection, maximum_selection, my_placement, select_action >::set_item_shown(), gui2::pane::set_origin_children(), gui2::dialogs::theme_list::set_selected_index(), gui2::dialogs::simple_item_selector::set_selected_index(), editor::location_palette::set_start_item(), editor::editor_palette< overlay >::set_start_item(), attack_type::special_active(), ng::connect_engine::start_game(), editor::context_manager::switch_context(), tod_manager::times(), theme::action::tooltip(), gui2::dialogs::mp_create_game::update_games_list(), and game_events::WML_HANDLER_FUNCTION().

utf8::string & utf8::insert ( utf8::string str,
const size_t  pos,
const utf8::string insert 
)
utf8::string utf8::lowercase ( const utf8::string s)
size_t utf8::size ( const utf8::string str)

Length in characters of a UTF-8 string.

Definition at line 86 of file unicode.cpp.

References byte_size_from_utf8_first(), ERR_GENERAL, and i.

Referenced by wb::action::action(), plugins_manager::any_running(), wb::attack::attack(), BOOST_AUTO_TEST_CASE(), gui2::implementation::builder_size_lock::build(), gui2::implementation::builder_drawing::build(), gui2::spacer::calculate_best_size(), gui2::matrix::calculate_best_size(), gui2::grid::calculate_best_size(), team::calculate_enemies(), font::pango_text::calculate_size(), ai::cfun_ai_get_recruitment_pattern(), events::menu_handler::change_side(), team::shroud_map::clear(), gui2::grid_implementation::column_request_reduce_width(), gui2::scrollbar_container::content_resize_request(), gui2::dialogs::chat_log::model::count_of_pages(), CVideo::current_resolution(), cut_surface(), do_replay_handle(), editor::location_palette::draw_contents(), editor::editor_palette< Item >::draw_contents(), wb::draw_numbers(), editor::editor_display::draw_sidebar(), font::draw_text(), DumpString(), erase(), wfl::builtins::debug_function::execute(), wfl::builtins::if_function::execute(), wfl::builtins::switch_function::execute(), wfl::builtins::debug_print_function::execute(), wfl::builtins::debug_profile_function::execute(), wfl::builtins::tomap_function::execute(), wfl::builtins::substring_function::execute(), wfl::builtins::replace_function::execute(), wfl::builtins::log_function::execute(), wfl::builtins::choose_function::execute(), wfl::builtins::sort_function::execute(), wfl::builtins::filter_function::execute(), wfl::builtins::find_function::execute(), wfl::builtins::map_function::execute(), wfl::builtins::sum_function::execute(), wfl::builtins::head_function::execute(), wfl::builtins::tail_function::execute(), editor::editor_controller::execute_command(), combatant::fight(), filesystem::file_size(), wfl::formula_function_expression::formula_function_expression(), game_launcher::game_launcher(), gui2::styled_widget::get_best_text_size(), wfl::function_symbol_table::get_builtins(), gui2::tree_view_node::get_current_size(), filesystem::get_files_in_dir(), gui2::tree_view_node::get_folded_size(), font::sdl_ttf::get_font(), wfl::builtins::zip_function::get_input(), team::get_side_color_index(), get_system_username(), gui2::tree_view_node::get_unfolded_size(), growstack(), gui2::text_box_base::handle_editing(), campaignd::server::handle_request_campaign(), filesystem::ifs_read(), impl_unit_attacks_len(), display::init_flags_for_side_internal(), gui2::password_box::insert_char(), gui2::text_box_base::insert_char(), font::pango_text::insert_text(), intf_describe_plugins(), game_lua_kernel::intf_find_cost_map(), game_lua_kernel::intf_find_path(), game_lua_kernel::intf_find_reach(), game_lua_kernel::intf_get_fog_or_shroud(), game_lua_kernel::intf_get_side_variable(), game_lua_kernel::intf_get_starting_location(), game_lua_kernel::intf_is_enemy(), game_lua_kernel::intf_match_side(), game_lua_kernel::intf_print(), game_lua_kernel::intf_put_recall_unit(), lua_fileops::intf_read_file(), game_lua_kernel::intf_set_side_variable(), game_lua_kernel::intf_set_village_owner(), spirit_po::po_message::is_plural(), gui2::window_implementation::layout(), gui2::grid::layout(), gui2::window::layout(), gui2::window::layout_linked_widgets(), LoadString(), luaD_growstack(), luaL_loadbufferx(), luaS_resize(), luaZ_fill(), hotkey::hotkey_keyboard::matches_helper(), map_location::matches_range(), plugins_manager::notify_event(), filesystem::ofs_write(), font::open_font_impl(), utils::operator<<(), packint(), team::shroud_map::place(), gui2::policy::placement::table::place(), plugins_manager::play_slice(), turn_info::process_network_data(), propagatemark(), ucs4_convert_impl::utf8_impl::read(), gui2::container_base::reduce_height(), gui2::grid::reduce_height(), gui2::container_base::reduce_width(), gui2::grid::reduce_width(), gui2::grid::request_placement(), gui2::container_base::request_reduce_height(), gui2::scrollbar_container::request_reduce_height(), gui2::styled_widget::request_reduce_height(), gui2::grid::request_reduce_height(), gui2::container_base::request_reduce_width(), gui2::scrollbar_container::request_reduce_width(), gui2::styled_widget::request_reduce_width(), gui2::grid::request_reduce_width(), font::text_cache::resize(), gui2::tree_view::resize_content(), gui2::listbox::resize_content(), editor::map_fragment::rotate_60_ccw(), editor::map_fragment::rotate_60_cw(), gui2::grid_implementation::row_request_reduce_height(), preferences::save_sound_buffer_size(), wb::manager::save_temp_move(), screen_area(), wb::manager::send_network_data(), CVideo::set_help_string(), gui2::dialogs::file_dialog::set_input_text(), editor::mouse_action_item::set_item_mouse_overlay(), gui2::widget::set_layout_size(), editor::mouse_action_map_label::set_mouse_overlay(), editor::mouse_action_village::set_mouse_overlay(), editor::mouse_action_select::set_mouse_overlay(), editor::mouse_action_paste::set_mouse_overlay(), editor::mouse_action_starting_position::set_mouse_overlay(), display::set_playing_team(), game_display::set_playing_team(), gui2::size_lock::set_target_size(), display::set_team(), editor::mouse_action::set_terrain_mouse_overlay(), editor::mouse_action_unit::set_unit_mouse_overlay(), gui2::password_box::set_value(), setarrayvector(), campaign_controller::show_carryover_message(), help::show_help(), special_locations_len(), str_pack(), str_packsize(), str_unpack(), t_translation::string_to_layer_(), utils::detail::sv_insert_aligned(), tremove(), truncate_as_ucs4(), unpackint(), gui2::dialogs::game_cache_options::update_cache_size_display(), gui2::dialogs::chat_log::controller::update_view_from_model(), validate_side(), team::shroud_map::value(), game_events::WML_HANDLER_FUNCTION(), and font::word_wrap_text().

utf8::string & utf8::truncate ( utf8::string str,
const size_t  size 
)

Truncates a UTF-8 string to the specified number of characters.

Parameters
strUTF-8 encoded string.
sizeSize to truncate to.
Note
This implementation does not check for valid UTF-8. Don't use it for user input.

Definition at line 117 of file unicode.cpp.

References erase().

Referenced by BOOST_AUTO_TEST_CASE(), utils::ellipsis_truncate(), and font::pango_text::set_maximum_length().

void utf8::truncate_as_ucs4 ( utf8::string str,
const size_t  size 
)

Truncates a UTF-8 string to the specified number of characters.

If the string has more than size UTF-8 characters it will be truncated to this size.

The output is guaranteed to be valid UTF-8.

Parameters
[in,out]str[in] String encoded in UTF-8. [out] String encoded UTF-8 that contains at most size codepoints.
sizeThe size to truncate to.

Definition at line 122 of file unicode.cpp.

References size(), and unicode_cast().

Referenced by wesnothd::chat_message::truncate_message().